-- Logs begin at Tue 2022-07-19 14:49:41 UTC, end at Tue 2022-07-19 16:15:57 UTC. -- Jul 19 15:15:46.178281 dv-cinder-849846-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 19 15:15:46.189219 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 19 15:15:46.189747 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 19 15:15:46 2022] *** Jul 19 15:15:46.189747 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 19 15:15:46.189747 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 19 15:15:46.189747 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: nodename: dv-cinder-849846-2-iscsi Jul 19 15:15:46.189747 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: machine: x86_64 Jul 19 15:15:46.189747 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: clock source: unix Jul 19 15:15:46.189747 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: pcre jit disabled Jul 19 15:15:46.189747 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: detected number of CPU cores: 4 Jul 19 15:15:46.189747 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: current working directory: / Jul 19 15:15:46.190206 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: detected binary path: /usr/bin/uwsgi-core Jul 19 15:15:46.190206 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: your processes number limit is 47680 Jul 19 15:15:46.190206 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: your memory page size is 4096 bytes Jul 19 15:15:46.190206 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: detected max file descriptor number: 2048 Jul 19 15:15:46.190206 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: lock engine: pthread robust mutexes Jul 19 15:15:46.190206 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: thunder lock: enabled Jul 19 15:15:46.190206 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 19 15:15:46.190206 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 19 15:15:46.234468 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: Python main interpreter initialized at 0x56386ca919c0 Jul 19 15:15:46.234468 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: python threads support enabled Jul 19 15:15:46.234468 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: your server socket listen backlog is limited to 100 connections Jul 19 15:15:46.234468 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: your mercy for graceful operations on workers is 90 seconds Jul 19 15:15:46.234843 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: mapped 403077 bytes (393 KB) for 2 cores Jul 19 15:15:46.234921 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: *** Operational MODE: preforking *** Jul 19 15:15:46.235168 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: *** uWSGI is running in multiple interpreter mode *** Jul 19 15:15:46.235168 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: spawned uWSGI master process (pid: 80257) Jul 19 15:15:46.235371 dv-cinder-849846-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 19 15:15:46.236480 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: spawned uWSGI worker 1 (pid: 80270, cores: 1) Jul 19 15:15:46.236480 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: spawned uWSGI worker 2 (pid: 80271, cores: 1) Jul 19 15:15:46.236480 dv-cinder-849846-2-iscsi devstack@keystone.service[80257]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 19 15:15:51.710194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 15:15:51.710194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: from cryptography.utils import int_from_bytes Jul 19 15:15:51.710940 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 15:15:51.710940 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: from cryptography.utils import int_from_bytes Jul 19 15:15:51.713255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 15:15:51.713255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: from cryptography.utils import int_from_bytes Jul 19 15:15:51.713718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 15:15:51.713718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: from cryptography.utils import int_from_bytes Jul 19 15:15:51.760350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 19 15:15:51.760492 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 19 15:15:51.760579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 19 15:15:51.760658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 19 15:15:51.760765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] command line args: [] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 19 15:15:51.760897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] command line args: [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 19 15:15:51.761026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 19 15:15:51.761123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 19 15:15:51.761236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ================================================================================ {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 19 15:15:51.761317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ================================================================================ {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 19 15:15:51.761478 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] admin_token = **** {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.761596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] admin_token = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.761727 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] config_dir = [] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.761851 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] config_dir = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.761929 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.762045 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.762135 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] config_source = [] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.762234 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] config_source = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.762316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] debug = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.762419 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] debug = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.762498 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.762765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.762907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.762983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.763078 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.763155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.763354 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.763439 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.763757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.763864 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.763933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.763996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] list_limit = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.764065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_config_append = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.764124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] log_config_append = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.764186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.764243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.764432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_dir = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.764534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] log_dir = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.764604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_file = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.764664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] log_file = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.764751 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.764848 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.765003 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.765116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.765275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.765339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.765522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.765610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.765707 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.765827 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.765899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.765966 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.766124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.766210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.766372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.766461 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.766608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.766690 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.766844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.766918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.767080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.767166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.767316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.767388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.767567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.767650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.767818 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.767908 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.768100 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.768176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.768602 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.768686 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.768802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] publish_errors = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.768914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] publish_errors = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.769028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.769185 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.769305 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.769388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.769501 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.769574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.769681 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.769802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.769918 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.770074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.770157 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] standard_threads = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.770348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] standard_threads = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.770427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.770610 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.770692 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.770804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.770878 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.770958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] use_journal = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.771081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.771158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] use_json = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.771283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] use_stderr = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.771367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] use_journal = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.771461 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] use_syslog = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.771548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] use_json = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.771683 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] use_stderr = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.771759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.771895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] use_syslog = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.771972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.772156 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 15:15:51.772247 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.772425 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.772500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.772669 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.772745 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.772925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.773100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.773208 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.773410 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.773487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.773659 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.773814 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.773903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.774101 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.774178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.774394 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.774503 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.774633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.774785 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.774886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.775722 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.775813 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.776000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.776126 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.776267 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.776390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.776569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.776646 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.776831 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.776918 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.777119 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.777233 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.777395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.777481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.777639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.777723 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.777901 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.777982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.778168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.778245 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.778433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.778592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.778678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.778866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.778943 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.779116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.779192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.779620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.779708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.779861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.780090 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.780180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.780355 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.780442 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.780589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.780785 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.780869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.781061 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.781158 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.781341 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.781468 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.781586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.781747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.781831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.782064 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.782150 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.782306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.782388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.782545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.782653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.782781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.782936 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.783020 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.783183 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.783266 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.783469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.783550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.783699 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.783773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.783951 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.784067 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.784187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.784270 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.784420 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.784497 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.784653 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.784729 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.784891 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.784972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.785127 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.785224 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.785381 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.785464 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.785622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.785699 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.785859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.785937 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.786099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.786201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.786341 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.786450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.786570 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.786668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.786798 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.786900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.787031 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.787204 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.787330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.787447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.787583 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.787693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.787805 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.787943 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.788062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.788207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.788315 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.788433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.788572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.788684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.788833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.788948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.789053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.789198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.789299 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.789409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.789545 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.789663 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.789804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.789913 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.790019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.790161 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.790269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.790387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.790532 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.790639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.791463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.791662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.791774 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.791922 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.792038 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.792115 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.792221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.792301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.792416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.792503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.792576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.792652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.792722 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.792825 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.792902 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.792981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.793053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.793162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.793243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.793415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.793526 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.793650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.793733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.793899 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.793984 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.794138 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.794216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.794376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.794454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.794620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.794696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.794860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.794942 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.795099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.795177 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.795334 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.795411 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.795570 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.795648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.795795 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.795875 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.796031 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.796110 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.796270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.796347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.796508 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.796597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.796742 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.796819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.796980 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.797070 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.797242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.797322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.797505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.797633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.797712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.797783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.797933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.798010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.798167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.798246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.798406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.798482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.798643 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.798728 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.external = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.798876 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.798953 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.799116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.799194 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.799353 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.799438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.799602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.799679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.password = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.799840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.799925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.token = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.800096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.800181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.800339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.800429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.800589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.800666 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.800823 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.800899 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] auth.external = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.801052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.801171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.801340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.801428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.801578 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.801656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.801810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] auth.password = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.801895 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.802042 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] auth.token = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.802127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.802280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.802366 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.802518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.802604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.802765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.802842 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.802994 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.803070 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.803239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.803328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.803483 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.803560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.803715 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.803794 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.803951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] credential.caching = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.804037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.804198 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.804276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.804447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.804526 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.804695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.804781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.804931 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.805017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.805191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.805273 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.805430 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.805524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.805692 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.805770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.805946 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.806023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.806185 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.806271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.806468 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.806550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.806705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.806825 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.806958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.807076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.807198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.807346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.807456 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.807593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.807705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.807852 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.807965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.808090 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] federation.caching = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.808213 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.808327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.808466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.808576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.808707 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.808822 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.808961 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.809044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.809237 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.809330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.809483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.809561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.809724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.809802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.809979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.810060 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.810215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.810292 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.810458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.810535 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.810701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.810810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.810938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity.caching = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.811069 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.811191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.811309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.811441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.811547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.811677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.811784 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.811913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.812037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.812273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.812352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.812509 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.812594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.812752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.812831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.813004 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.813123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.813280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.813360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.813517 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.813595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.813752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.813835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.813986 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.814064 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.814231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.814314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.814471 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.814551 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.814713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.814791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.814959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.815038 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.815205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.815283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.815458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.815542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.815696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.815775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.815934 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.816012 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.816166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.816245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.816405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.816483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.816637 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.816717 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.816873 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.816952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.817113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.817205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.817375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.817459 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.817611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.817687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.817844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.817929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.818085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.818163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.818331 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.818407 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.818566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.818644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.818806 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.818884 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.819049 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.819128 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.819281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.819358 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.819519 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.819598 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.819751 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.819827 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.819986 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.820062 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.820223 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.820302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.820470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.820555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.820708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.820785 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.820941 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.821019 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.821226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.821310 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.821440 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.821519 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.821670 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.821748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.821900 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.821978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.822195 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.822310 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.822617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.822694 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.822884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.822961 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.823123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.823208 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.823357 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.823434 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.823591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.823668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.823828 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.823912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.824070 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.824149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.824305 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.824383 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.824550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.824634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.824798 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.824876 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.825045 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.825122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.825307 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.825450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.825576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.825699 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.825822 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.825946 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.826069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.826199 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.826339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.826464 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.826587 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.826709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.826831 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.826961 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.827085 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.827208 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.827395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.827517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.827640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.827771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.827895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.828029 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.828152 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.828275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.828396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.828526 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.828658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.828781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.828903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.829026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.829197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.829331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.829458 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.829586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.829712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.829847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.829972 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.830106 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.830232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.830356 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.830479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.830604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.830731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.830868 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.830994 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.831119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.831244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.831368 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.831491 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.831632 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.831757 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.831880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.832004 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.832129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.832252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.832387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.832513 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.832636 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.832762 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.832886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.833009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] resource.caching = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.833158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.833283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.833424 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] role.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.833547 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.833670 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] role.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.833793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.833989 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.834114 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.834237 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.834359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.834481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.834604 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.834734 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.834858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.834979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.835112 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.835235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.835366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.835366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] role.caching = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.835597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.835719 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] role.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.835841 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.835963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.836092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.836216 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.836338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.836459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.836581 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.836702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.836840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.836964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.837086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.837260 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.837389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.837515 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.838152 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.838277 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.838403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.838529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.838658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.838785 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.838919 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.839054 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.839177 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.839302 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.839302 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.839541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.839674 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.839800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.839800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.840037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.840163 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.840288 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.840421 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.840545 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.840679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.840805 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.840929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.841052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.841202 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.841327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.841450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.841571 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.841571 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.841809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.841932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.842054 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.842177 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.842313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.842436 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.842566 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.842689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.842812 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.842933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.843055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.843177 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.843307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.843430 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.843552 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.843674 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.843797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.843919 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.844058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.844181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.844303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.844431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.844553 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.844675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] token.caching = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.844805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.844928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.845050 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.845215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.845340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.845466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.845600 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.845735 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.845861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.845985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.846109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.846233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.846364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.846491 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.846615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.846738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.846864 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.846989 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.847120 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.847246 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.847370 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.847506 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.847630 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.connection = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.847764 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.847897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.848022 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.848155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.848280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.848404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.848528 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.848658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.848789 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.848915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.849039 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.849180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.849313 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.connection = **** {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.849446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.849579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.849703 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.849833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.849957 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.850080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.850203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.850325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.850447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.850577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.850700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.850823 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.850947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.851078 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.851200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.851330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.851454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.851576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.851699 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.851821 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.851944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.852074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.852074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.852304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.852304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 19 15:15:51.852534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.852534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.852534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.852534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.852534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.852534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 15:15:51.852534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80271) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 19 15:15:51.881635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:51.881863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:51.881863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:51.882124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:51.882255 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:51.882387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:51.891074 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:51.891221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:51.919965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:51.920158 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:51.920289 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:51.920430 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.062184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.063003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.112154 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.112392 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.112702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.112941 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.177602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.177826 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.177961 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.178112 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.178246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.178377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.178506 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.178631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.178761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.178910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.179040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.179040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.204587 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.204817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.205005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.205191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.205350 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.205503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.205653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.205825 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.205974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.206121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.206266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.206411 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.206555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.206737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.217164 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.217324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.230388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.230617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.230786 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.230927 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.231062 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.231204 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.231339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.231479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.231611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.231742 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.231872 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.232002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80271) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.232151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.232151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 15:15:52.250031 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.250031 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 15:15:52.250220 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.250286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.250966 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 15:15:52.251040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 15:15:52.251040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.251218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.251218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 15:15:52.251464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.251530 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.251589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.251649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.251708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.251767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.251767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.262021 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.262296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.262430 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.262559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.262709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.262837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.262974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.263254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.270921 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.271398 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.274225 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.274506 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.274719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.274893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.274987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.275715 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.275914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.276042 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.276203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.276294 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.276494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.276574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.276758 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.276837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.277029 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.277117 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.277330 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.277411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.277605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.277684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.277881 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.278150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.304810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.305532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.306481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.306777 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.307178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.307461 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.311964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.312290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.312524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.312740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.313022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.313327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.319738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.320019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.320592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.320877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.324747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.325033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.325331 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.325609 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.325690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.325877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.325959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.326142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.326223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.326403 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.326485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.326667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.326747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.327023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.327295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.327561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.345804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.346080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.346727 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.347005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.350652 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.351670 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.355042 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.355335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.355867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.356162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.360304 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.361223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.363932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.364836 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.367676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.367972 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.368253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.368541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.368694 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.368861 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.369024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.369283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.369373 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.369627 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.369922 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.370203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.388246 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.388620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.388888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.389050 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.389159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.389457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.389733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.390005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.397012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.397993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.398736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.399010 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.399714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.399993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.402880 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.403350 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.403631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.403860 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.403945 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.404145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.404228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.404679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.404960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.405266 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.418343 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.418621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.418898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.419190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.419280 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.419451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.419535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.419738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.419821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.420012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.420103 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.420310 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.420608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.420899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.439149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.440184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.442416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.443511 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.445715 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.446853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.447418 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.448636 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.449010 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.449307 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.449591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.450302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.450583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.450875 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.456965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.458315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.458588 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.459933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.461833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.462106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.462385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.463233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.464018 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.464311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.464595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.464872 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.472412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.472699 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.472975 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.474103 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.474396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.474679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.481020 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80271) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.481326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.481604 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.481879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.482149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.482433 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.482699 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.482850 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 15:15:52.482988 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.483115 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.483234 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.483353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.483479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.483596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.483744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.483877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.484069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.484188 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.484303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.484416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.484530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80271) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.484678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.484822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.485105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.485419 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.485684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.485955 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.486221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 15:15:52.523203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56386ca919c0 pid: 80271 (default app) Jul 19 15:15:52.524892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56386ca919c0 pid: 80270 (default app) Jul 19 15:15:52.526664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e69c3-954b-42f5-8e8a-e6ae97f3d4e4 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:15:52.526960 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e69c3-954b-42f5-8e8a-e6ae97f3d4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:15:52.527242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e69c3-954b-42f5-8e8a-e6ae97f3d4e4 None None] PATH_INFO: `/v3` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:15:52.531711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Tue Jul 19 15:15:52 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:00.192579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183c863-6d51-442d-b71e-f1db1b2a718d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:00.193164 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183c863-6d51-442d-b71e-f1db1b2a718d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:00.193164 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183c863-6d51-442d-b71e-f1db1b2a718d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:00.197645 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:16:00.202800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f89ffb-24b2-4b2e-9726-8ea3b553363a None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:00.203235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f89ffb-24b2-4b2e-9726-8ea3b553363a None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:00.203361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f89ffb-24b2-4b2e-9726-8ea3b553363a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:00.273829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: INFO dbcounter [None req-e0f89ffb-24b2-4b2e-9726-8ea3b553363a None None] Registered counter for database keystone Jul 19 15:16:00.296267 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG oslo_db.sqlalchemy.engines [None req-e0f89ffb-24b2-4b2e-9726-8ea3b553363a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80271) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 15:16:00.302389 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writer thread running {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 15:16:00.480247 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG passlib.handlers.bcrypt [None req-e0f89ffb-24b2-4b2e-9726-8ea3b553363a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80271) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 19 15:16:00.480672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG passlib.handlers.bcrypt [None req-e0f89ffb-24b2-4b2e-9726-8ea3b553363a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80271) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 19 15:16:00.535343 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-e0f89ffb-24b2-4b2e-9726-8ea3b553363a None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:00.536964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e0f89ffb-24b2-4b2e-9726-8ea3b553363a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:00.659322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 19 15:16:00.842063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183c863-6d51-442d-b71e-f1db1b2a718d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:00.911982 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: INFO dbcounter [None req-1183c863-6d51-442d-b71e-f1db1b2a718d None None] Registered counter for database keystone Jul 19 15:16:00.932513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG oslo_db.sqlalchemy.engines [None req-1183c863-6d51-442d-b71e-f1db1b2a718d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80270) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 15:16:00.938728 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writer thread running {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 15:16:01.058995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183c863-6d51-442d-b71e-f1db1b2a718d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:01.063262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300f62a-7120-442f-948b-b997ecd2a118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:01.063920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300f62a-7120-442f-948b-b997ecd2a118 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:01.064207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300f62a-7120-442f-948b-b997ecd2a118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:01.064486 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300f62a-7120-442f-948b-b997ecd2a118 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:01.072641 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300f62a-7120-442f-948b-b997ecd2a118 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:01.124169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1300f62a-7120-442f-948b-b997ecd2a118 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 19 15:16:01.124455 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300f62a-7120-442f-948b-b997ecd2a118 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:01.134246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Tue Jul 19 15:16:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:16:01.140175 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f89ffb-24b2-4b2e-9726-8ea3b553363a None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:01.154631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f89ffb-24b2-4b2e-9726-8ea3b553363a None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:01.159003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131781f2-39c2-4f44-a2db-8b1a11186a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:01.160125 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131781f2-39c2-4f44-a2db-8b1a11186a65 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:01.160413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131781f2-39c2-4f44-a2db-8b1a11186a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:01.160687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131781f2-39c2-4f44-a2db-8b1a11186a65 admin admin] PATH_INFO: `/v3/projects/5bb6648f966d4104a0136e3bb203cb76` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:01.162544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131781f2-39c2-4f44-a2db-8b1a11186a65 admin admin] RBAC: Authorizing `identity:get_project(project_id=5bb6648f966d4104a0136e3bb203cb76)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:01.215782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: WARNING keystone.common.rbac_enforcer.enforcer [None req-131781f2-39c2-4f44-a2db-8b1a11186a65 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 19 15:16:01.216359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131781f2-39c2-4f44-a2db-8b1a11186a65 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:01.220642 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Tue Jul 19 15:16:01 2022] GET /identity/v3/projects/5bb6648f966d4104a0136e3bb203cb76 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:03.261893 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7ba17-4de5-4169-81f3-69fe47c1082b None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:03.262328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7ba17-4de5-4169-81f3-69fe47c1082b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:03.262452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7ba17-4de5-4169-81f3-69fe47c1082b None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:03.266572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:16:03.271959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1e927-6fc9-4144-b82b-7fcadef73bca None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:03.272365 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1e927-6fc9-4144-b82b-7fcadef73bca None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:03.272532 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1e927-6fc9-4144-b82b-7fcadef73bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:03.292001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-d4e1e927-6fc9-4144-b82b-7fcadef73bca None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:03.293560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d4e1e927-6fc9-4144-b82b-7fcadef73bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:03.301394 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 19 15:16:03.324464 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7ba17-4de5-4169-81f3-69fe47c1082b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:03.336310 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7ba17-4de5-4169-81f3-69fe47c1082b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:03.340589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ad604-2df3-40d8-857f-8c81754faaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:03.341246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ad604-2df3-40d8-857f-8c81754faaa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:03.341536 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ad604-2df3-40d8-857f-8c81754faaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:03.341814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ad604-2df3-40d8-857f-8c81754faaa0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:03.349562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ad604-2df3-40d8-857f-8c81754faaa0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:03.351351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ad604-2df3-40d8-857f-8c81754faaa0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:03.360959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Tue Jul 19 15:16:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:16:03.366728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1e927-6fc9-4144-b82b-7fcadef73bca None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:03.376448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1e927-6fc9-4144-b82b-7fcadef73bca None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:03.380774 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca7aae-8c3a-47fb-80a6-b9866158bf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:03.381918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca7aae-8c3a-47fb-80a6-b9866158bf53 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:03.382054 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca7aae-8c3a-47fb-80a6-b9866158bf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:03.382239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca7aae-8c3a-47fb-80a6-b9866158bf53 admin admin] PATH_INFO: `/v3/users/669959f6d82b46b1b551b6c19bf21c50` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:03.383668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca7aae-8c3a-47fb-80a6-b9866158bf53 admin admin] RBAC: Authorizing `identity:get_user(user_id=669959f6d82b46b1b551b6c19bf21c50)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:03.385504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca7aae-8c3a-47fb-80a6-b9866158bf53 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:03.387083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Tue Jul 19 15:16:03 2022] GET /identity/v3/users/669959f6d82b46b1b551b6c19bf21c50 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:05.465311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a6963-f27b-46ed-8138-c72c301e870c None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:05.465843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a6963-f27b-46ed-8138-c72c301e870c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:05.465843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a6963-f27b-46ed-8138-c72c301e870c None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:05.467227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:05.472397 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81951d80-784e-4111-88cb-d2d8145257c6 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:05.472826 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81951d80-784e-4111-88cb-d2d8145257c6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:05.472957 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81951d80-784e-4111-88cb-d2d8145257c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:05.489935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-81951d80-784e-4111-88cb-d2d8145257c6 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:05.491121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-81951d80-784e-4111-88cb-d2d8145257c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:05.502362 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cbeb4-7348-4d8e-88ec-6491a6664718 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:05.502588 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cbeb4-7348-4d8e-88ec-6491a6664718 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:05.502867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cbeb4-7348-4d8e-88ec-6491a6664718 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:05.504204 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:16:05.509714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb7069-d5c1-44f5-aee5-80d739f0ae81 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:05.510001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb7069-d5c1-44f5-aee5-80d739f0ae81 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:05.510277 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb7069-d5c1-44f5-aee5-80d739f0ae81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:05.559033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG passlib.handlers.bcrypt [None req-efdb7069-d5c1-44f5-aee5-80d739f0ae81 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80270) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 19 15:16:05.559524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG passlib.handlers.bcrypt [None req-efdb7069-d5c1-44f5-aee5-80d739f0ae81 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80270) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 19 15:16:05.576120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:05.599772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81951d80-784e-4111-88cb-d2d8145257c6 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:05.606156 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-efdb7069-d5c1-44f5-aee5-80d739f0ae81 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:05.607747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-efdb7069-d5c1-44f5-aee5-80d739f0ae81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:05.623441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:05.705440 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81951d80-784e-4111-88cb-d2d8145257c6 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:05.708891 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:05.730889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb7069-d5c1-44f5-aee5-80d739f0ae81 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:05.742167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e1125-5e58-4073-96ff-12296d87007b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:05.743594 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e1125-5e58-4073-96ff-12296d87007b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:05.743941 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e1125-5e58-4073-96ff-12296d87007b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:05.744353 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e1125-5e58-4073-96ff-12296d87007b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:05.755500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e1125-5e58-4073-96ff-12296d87007b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-451e1125-5e58-4073-96ff-12296d87007b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:05.767710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:05.768617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:05.769466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:05.769466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:05.769466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:05.769466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:05.769466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:05.769466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:05.769466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:05.769466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:05.769466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:05.769466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:05.769466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:05.769466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:05.769466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:16:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:16:05.775663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e1125-5e58-4073-96ff-12296d87007b None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:05.835977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb7069-d5c1-44f5-aee5-80d739f0ae81 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:05.838429 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:05.859147 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:05.874301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836338e-20fa-4519-9cf4-65402b673f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:05.875346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836338e-20fa-4519-9cf4-65402b673f06 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:05.875634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836338e-20fa-4519-9cf4-65402b673f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:05.875911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836338e-20fa-4519-9cf4-65402b673f06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:05.877734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e1125-5e58-4073-96ff-12296d87007b None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:05.911088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24edc9e-9360-4be8-a868-a87714c78e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:05.911746 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24edc9e-9360-4be8-a868-a87714c78e0a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:05.912031 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24edc9e-9360-4be8-a868-a87714c78e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:05.912310 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24edc9e-9360-4be8-a868-a87714c78e0a None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:05.913061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24edc9e-9360-4be8-a868-a87714c78e0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:05.913401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24edc9e-9360-4be8-a868-a87714c78e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:05.914846 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24edc9e-9360-4be8-a868-a87714c78e0a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:05.926564 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:05.932860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24edc9e-9360-4be8-a868-a87714c78e0a None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:05.943532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836338e-20fa-4519-9cf4-65402b673f06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:05.945108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836338e-20fa-4519-9cf4-65402b673f06 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:06.033015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24edc9e-9360-4be8-a868-a87714c78e0a None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:06.042862 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 8/16] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 15:16:06.048587 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836338e-20fa-4519-9cf4-65402b673f06 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:06.051245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:06.067991 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc56ad6-3c5a-4177-819a-4ef3e9066815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:06.068868 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc56ad6-3c5a-4177-819a-4ef3e9066815 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:06.069169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc56ad6-3c5a-4177-819a-4ef3e9066815 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:06.069462 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc56ad6-3c5a-4177-819a-4ef3e9066815 None admin] PATH_INFO: `/v3/users/669959f6d82b46b1b551b6c19bf21c50` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:06.070667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc56ad6-3c5a-4177-819a-4ef3e9066815 None admin] RBAC: Authorizing `identity:get_user(user_id=669959f6d82b46b1b551b6c19bf21c50)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:06.072869 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc56ad6-3c5a-4177-819a-4ef3e9066815 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:06.074714 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:05 2022] GET /identity/v3/users/669959f6d82b46b1b551b6c19bf21c50 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:06.079776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc56ad6-3c5a-4177-819a-4ef3e9066815 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:06.094359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:06.148993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836338e-20fa-4519-9cf4-65402b673f06 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:06.179285 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc56ad6-3c5a-4177-819a-4ef3e9066815 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:06.185193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd00c50-9843-46ef-938f-1e0077b8924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:06.185989 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd00c50-9843-46ef-938f-1e0077b8924d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:06.186270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd00c50-9843-46ef-938f-1e0077b8924d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:06.186545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd00c50-9843-46ef-938f-1e0077b8924d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:06.206156 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd00c50-9843-46ef-938f-1e0077b8924d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:06.207619 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd00c50-9843-46ef-938f-1e0077b8924d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:06.213003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a81b43-15c2-4f40-9534-ffdc0b5e4fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:06.213856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a81b43-15c2-4f40-9534-ffdc0b5e4fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:06.214137 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a81b43-15c2-4f40-9534-ffdc0b5e4fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:06.214425 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a81b43-15c2-4f40-9534-ffdc0b5e4fc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:06.215600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a81b43-15c2-4f40-9534-ffdc0b5e4fc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:06.217023 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a81b43-15c2-4f40-9534-ffdc0b5e4fc9 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:06.218786 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:06.224438 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a81b43-15c2-4f40-9534-ffdc0b5e4fc9 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:06.228252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-2bd00c50-9843-46ef-938f-1e0077b8924d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:16:06.228252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:06.228252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:06.228252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:06.228252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:06.228252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:06.228252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:06.228252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:06.228252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:06.228252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:06.228252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:06.228252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:06.228252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:16:06.228252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:16:06.228252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:06.229254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:06.230174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:16:06.230174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:16:06.230174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:16:06.230174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:06.230174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:06.235292 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd00c50-9843-46ef-938f-1e0077b8924d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:06.293115 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:06.311915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:06.330909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a81b43-15c2-4f40-9534-ffdc0b5e4fc9 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:06.336923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd00c50-9843-46ef-938f-1e0077b8924d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:06.364815 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa64f5-e7cb-4a79-b129-8b3b9e368aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:06.365494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa64f5-e7cb-4a79-b129-8b3b9e368aef None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:06.365776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa64f5-e7cb-4a79-b129-8b3b9e368aef None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:06.366056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa64f5-e7cb-4a79-b129-8b3b9e368aef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:06.366888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa64f5-e7cb-4a79-b129-8b3b9e368aef None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.domain.id=default, user.id=669959f6d82b46b1b551b6c19bf21c50) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:06.367197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa64f5-e7cb-4a79-b129-8b3b9e368aef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:06.369260 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa64f5-e7cb-4a79-b129-8b3b9e368aef None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:06.373457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67891b6-bd9f-4689-97a7-7028b13663f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:06.374127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67891b6-bd9f-4689-97a7-7028b13663f9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:06.374420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67891b6-bd9f-4689-97a7-7028b13663f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:06.374703 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67891b6-bd9f-4689-97a7-7028b13663f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:06.375442 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67891b6-bd9f-4689-97a7-7028b13663f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:06.375753 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67891b6-bd9f-4689-97a7-7028b13663f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:06.375888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1322 bytes} [Tue Jul 19 15:16:06 2022] GET /identity/v3/role_assignments?user.id=669959f6d82b46b1b551b6c19bf21c50&scope.domain.id=default&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:06.377166 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67891b6-bd9f-4689-97a7-7028b13663f9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:06.396906 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 10/21] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:08.793488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cba2f-5e5b-4e9c-bd91-df4a71895f92 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:08.794001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cba2f-5e5b-4e9c-bd91-df4a71895f92 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:08.794185 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cba2f-5e5b-4e9c-bd91-df4a71895f92 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:08.795407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:08.811488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4c35e-213a-4234-a278-a079f8888ecd None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:08.811997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4c35e-213a-4234-a278-a079f8888ecd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:08.812072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4c35e-213a-4234-a278-a079f8888ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:08.838166 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-27a4c35e-213a-4234-a278-a079f8888ecd None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:08.839397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-27a4c35e-213a-4234-a278-a079f8888ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:08.939489 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 11/23] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:08.978703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101cba2f-5e5b-4e9c-bd91-df4a71895f92 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:08.980728 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533e64e-e16d-4e6a-bcc3-f102f68cada8 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:08.981029 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533e64e-e16d-4e6a-bcc3-f102f68cada8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:08.981339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533e64e-e16d-4e6a-bcc3-f102f68cada8 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:08.982874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 12/24] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:16:08.987904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933d6a0-58c8-461a-b4b2-271149464e16 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:08.988201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933d6a0-58c8-461a-b4b2-271149464e16 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:08.988488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933d6a0-58c8-461a-b4b2-271149464e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:09.013464 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1933d6a0-58c8-461a-b4b2-271149464e16 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:09.015824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1933d6a0-58c8-461a-b4b2-271149464e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:09.133048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:09.176349 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 13/25] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:09.177520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa74837-1bf2-47c6-8c70-1c1f6b28cab8 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:09.178067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa74837-1bf2-47c6-8c70-1c1f6b28cab8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:09.178577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa74837-1bf2-47c6-8c70-1c1f6b28cab8 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:09.180059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 14/26] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:09.181562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd888984-3b35-40fc-8439-621343f6c8a8 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:09.182060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd888984-3b35-40fc-8439-621343f6c8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:09.182556 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd888984-3b35-40fc-8439-621343f6c8a8 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:09.183308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101cba2f-5e5b-4e9c-bd91-df4a71895f92 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:09.184260 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 15/27] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:09.189791 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c1968-fdad-4309-a971-43398451788d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:09.190319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c1968-fdad-4309-a971-43398451788d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:09.190822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c1968-fdad-4309-a971-43398451788d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:09.218161 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-538c1968-fdad-4309-a971-43398451788d None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:09.219719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-538c1968-fdad-4309-a971-43398451788d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:09.223679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33370b-6cf3-4c16-bf0a-e7424b63f4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:09.224499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33370b-6cf3-4c16-bf0a-e7424b63f4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:09.224785 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33370b-6cf3-4c16-bf0a-e7424b63f4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:09.225063 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33370b-6cf3-4c16-bf0a-e7424b63f4e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:09.239120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33370b-6cf3-4c16-bf0a-e7424b63f4e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-1f33370b-6cf3-4c16-bf0a-e7424b63f4e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:09.248339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:09.249306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:09.250058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:09.250058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:09.250058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:09.250058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:09.250058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:09.250058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:09.250058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:09.250058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:09.250058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:09.250058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:09.250058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:09.250058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:09.252321 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 13/28] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:16:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:09.254346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f4692-8e2d-4ced-9f7d-a289c50d5041 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:09.254781 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f4692-8e2d-4ced-9f7d-a289c50d5041 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:09.255199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f4692-8e2d-4ced-9f7d-a289c50d5041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:09.264309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:09.280623 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-967f4692-8e2d-4ced-9f7d-a289c50d5041 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:09.282443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-967f4692-8e2d-4ced-9f7d-a289c50d5041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:09.308949 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 16/29] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:09.309989 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c1968-fdad-4309-a971-43398451788d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:09.365908 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 14/30] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:09.367112 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f4692-8e2d-4ced-9f7d-a289c50d5041 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:09.420932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c1968-fdad-4309-a971-43398451788d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:09.432712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:09.457486 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad036d-24c0-47f1-bef4-1ff50ba5c825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:09.458531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad036d-24c0-47f1-bef4-1ff50ba5c825 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:09.458990 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad036d-24c0-47f1-bef4-1ff50ba5c825 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:09.459431 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad036d-24c0-47f1-bef4-1ff50ba5c825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:09.471203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad036d-24c0-47f1-bef4-1ff50ba5c825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:09.472916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad036d-24c0-47f1-bef4-1ff50ba5c825 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:09.475772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f4692-8e2d-4ced-9f7d-a289c50d5041 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:09.493512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-0dad036d-24c0-47f1-bef4-1ff50ba5c825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:16:09.493512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:09.493512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:09.493512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:09.493512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:09.493512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:09.493512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:09.493512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:09.493512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:09.493512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:09.493512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:09.493512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:09.493512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:16:09.493512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:16:09.493512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:09.494447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:09.495236 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:16:09.495236 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:16:09.495236 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:16:09.495236 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:09.497203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 17/31] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:09.498238 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad036d-24c0-47f1-bef4-1ff50ba5c825 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:09.514390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190def6-232e-46bd-a6df-5617154ab69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:09.515171 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190def6-232e-46bd-a6df-5617154ab69c None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:09.515659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190def6-232e-46bd-a6df-5617154ab69c None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:09.516093 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190def6-232e-46bd-a6df-5617154ab69c None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:09.517011 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190def6-232e-46bd-a6df-5617154ab69c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:09.517505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190def6-232e-46bd-a6df-5617154ab69c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:09.519040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190def6-232e-46bd-a6df-5617154ab69c None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:09.520570 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:09.529218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 15/32] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:09.530361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190def6-232e-46bd-a6df-5617154ab69c None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:09.606847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad036d-24c0-47f1-bef4-1ff50ba5c825 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:09.636979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190def6-232e-46bd-a6df-5617154ab69c None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:09.642922 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57614093-4b6e-4771-8da0-5294f998294d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:09.643774 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57614093-4b6e-4771-8da0-5294f998294d None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:09.644219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57614093-4b6e-4771-8da0-5294f998294d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:09.644632 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57614093-4b6e-4771-8da0-5294f998294d None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:09.645779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57614093-4b6e-4771-8da0-5294f998294d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:09.647328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57614093-4b6e-4771-8da0-5294f998294d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:09.675896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5fe91-d4ab-4ef5-af24-2a0bac21931e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:09.676911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5fe91-d4ab-4ef5-af24-2a0bac21931e None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:09.677449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5fe91-d4ab-4ef5-af24-2a0bac21931e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:09.677911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5fe91-d4ab-4ef5-af24-2a0bac21931e None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:09.678847 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5fe91-d4ab-4ef5-af24-2a0bac21931e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:09.680984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5fe91-d4ab-4ef5-af24-2a0bac21931e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:09.779682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 18/33] 10.222.0.42 () {60 vars in 1131 bytes} [Tue Jul 19 15:16:09 2022] POST /identity/v3/roles => generated 219 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:16:09.781225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57614093-4b6e-4771-8da0-5294f998294d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:09.826747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:09.844423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 16/34] 10.222.0.42 () {60 vars in 1131 bytes} [Tue Jul 19 15:16:09 2022] POST /identity/v3/roles => generated 225 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:16:09.849966 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5fe91-d4ab-4ef5-af24-2a0bac21931e None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:10.000379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57614093-4b6e-4771-8da0-5294f998294d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:10.002764 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:10.050421 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78483912-aec1-43d7-9c89-9f8ed9fc87cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:10.051386 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78483912-aec1-43d7-9c89-9f8ed9fc87cd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:10.051817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78483912-aec1-43d7-9c89-9f8ed9fc87cd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:10.052255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78483912-aec1-43d7-9c89-9f8ed9fc87cd None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:10.053744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78483912-aec1-43d7-9c89-9f8ed9fc87cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:10.054219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78483912-aec1-43d7-9c89-9f8ed9fc87cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:10.056606 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78483912-aec1-43d7-9c89-9f8ed9fc87cd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:10.072463 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 19/35] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:10.078239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78483912-aec1-43d7-9c89-9f8ed9fc87cd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:10.123590 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5fe91-d4ab-4ef5-af24-2a0bac21931e None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:10.192780 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba417c-55a2-4f30-b99b-aec9bab0023a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:10.193718 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba417c-55a2-4f30-b99b-aec9bab0023a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:10.194005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba417c-55a2-4f30-b99b-aec9bab0023a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:10.194481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba417c-55a2-4f30-b99b-aec9bab0023a None admin] PATH_INFO: `/v3/users/669959f6d82b46b1b551b6c19bf21c50` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:10.199191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba417c-55a2-4f30-b99b-aec9bab0023a None admin] RBAC: Authorizing `identity:get_user(user_id=669959f6d82b46b1b551b6c19bf21c50)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:10.201327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba417c-55a2-4f30-b99b-aec9bab0023a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:10.201524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:10.203540 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 17/36] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:09 2022] GET /identity/v3/users/669959f6d82b46b1b551b6c19bf21c50 => generated 247 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:10.209214 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba417c-55a2-4f30-b99b-aec9bab0023a None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:10.250519 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78483912-aec1-43d7-9c89-9f8ed9fc87cd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:10.300400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de21a38-56c8-4498-ba7b-7392d7b4102a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:10.301082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de21a38-56c8-4498-ba7b-7392d7b4102a None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:10.301562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de21a38-56c8-4498-ba7b-7392d7b4102a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:10.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de21a38-56c8-4498-ba7b-7392d7b4102a None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:10.303158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de21a38-56c8-4498-ba7b-7392d7b4102a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:10.304799 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de21a38-56c8-4498-ba7b-7392d7b4102a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:10.344282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 20/37] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Jul 19 15:16:10 2022] POST /identity/v3/projects => generated 301 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:16:10.345830 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863161d3-99ab-4b25-8796-c466f4d93f06 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:10.346130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863161d3-99ab-4b25-8796-c466f4d93f06 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:10.346479 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863161d3-99ab-4b25-8796-c466f4d93f06 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:10.346572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba417c-55a2-4f30-b99b-aec9bab0023a None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:10.347983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 21/38] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:10.353803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989db72-d7bf-48d2-808c-6ea7a16a79d6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:10.354357 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989db72-d7bf-48d2-808c-6ea7a16a79d6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:10.354824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989db72-d7bf-48d2-808c-6ea7a16a79d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:10.374993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d989db72-d7bf-48d2-808c-6ea7a16a79d6 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:10.377023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d989db72-d7bf-48d2-808c-6ea7a16a79d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:10.382432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:10.394502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7934809-47b4-4e16-a036-21844ca2d027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:10.395684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7934809-47b4-4e16-a036-21844ca2d027 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:10.396232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7934809-47b4-4e16-a036-21844ca2d027 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:10.396659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7934809-47b4-4e16-a036-21844ca2d027 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:10.398593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7934809-47b4-4e16-a036-21844ca2d027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:10.401183 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7934809-47b4-4e16-a036-21844ca2d027 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:10.403493 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 18/39] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:10.415360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7934809-47b4-4e16-a036-21844ca2d027 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:10.463287 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 22/40] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:10.465010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2d300-618a-4eaa-816b-881f76b140db None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:10.465439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2d300-618a-4eaa-816b-881f76b140db None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:10.465935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2d300-618a-4eaa-816b-881f76b140db None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:10.467443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 23/41] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:10.473392 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3aa6f-eb1f-4d12-a55f-f17c56445cbc None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:10.473764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3aa6f-eb1f-4d12-a55f-f17c56445cbc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:10.474146 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3aa6f-eb1f-4d12-a55f-f17c56445cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:10.494561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4db3aa6f-eb1f-4d12-a55f-f17c56445cbc None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:10.495826 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4db3aa6f-eb1f-4d12-a55f-f17c56445cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:10.564091 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:10.578327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 24/42] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:10.579180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3aa6f-eb1f-4d12-a55f-f17c56445cbc None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:10.641946 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7934809-47b4-4e16-a036-21844ca2d027 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:10.675883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64ea2f-cc4c-498f-83b3-12065f52d20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:10.676759 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64ea2f-cc4c-498f-83b3-12065f52d20e None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:10.677053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64ea2f-cc4c-498f-83b3-12065f52d20e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:10.677360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64ea2f-cc4c-498f-83b3-12065f52d20e None admin] PATH_INFO: `/v3/domains/default/users/669959f6d82b46b1b551b6c19bf21c50/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:10.679397 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64ea2f-cc4c-498f-83b3-12065f52d20e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=669959f6d82b46b1b551b6c19bf21c50, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:10.680909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64ea2f-cc4c-498f-83b3-12065f52d20e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:10.683017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3aa6f-eb1f-4d12-a55f-f17c56445cbc None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:10.696679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 19/43] 10.222.0.42 () {58 vars in 1361 bytes} [Tue Jul 19 15:16:10 2022] PUT /identity/v3/domains/default/users/669959f6d82b46b1b551b6c19bf21c50/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:16:10.697628 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64ea2f-cc4c-498f-83b3-12065f52d20e None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:10.728023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e687f-b5b2-47a4-adab-d1f285c5b7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:10.728770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e687f-b5b2-47a4-adab-d1f285c5b7d1 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:10.729051 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e687f-b5b2-47a4-adab-d1f285c5b7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:10.729382 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e687f-b5b2-47a4-adab-d1f285c5b7d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:10.730160 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e687f-b5b2-47a4-adab-d1f285c5b7d1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:10.731615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e687f-b5b2-47a4-adab-d1f285c5b7d1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:10.746673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 25/44] 10.222.0.42 () {60 vars in 1131 bytes} [Tue Jul 19 15:16:10 2022] POST /identity/v3/roles => generated 223 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:16:10.773717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:10.835322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64ea2f-cc4c-498f-83b3-12065f52d20e None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:10.878778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2aad82-a6ae-4700-8e58-7e657a46d3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:10.879649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2aad82-a6ae-4700-8e58-7e657a46d3ce None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:10.879941 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2aad82-a6ae-4700-8e58-7e657a46d3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:10.880222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2aad82-a6ae-4700-8e58-7e657a46d3ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:10.881537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2aad82-a6ae-4700-8e58-7e657a46d3ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:10.883037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2aad82-a6ae-4700-8e58-7e657a46d3ce None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:10.885815 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 20/45] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:10.891557 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e687f-b5b2-47a4-adab-d1f285c5b7d1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:10.952092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:11.007755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e687f-b5b2-47a4-adab-d1f285c5b7d1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:11.041457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa2620-7448-49f3-90e9-92eab4c0013e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:11.042125 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa2620-7448-49f3-90e9-92eab4c0013e None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:11.042415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa2620-7448-49f3-90e9-92eab4c0013e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:11.042700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa2620-7448-49f3-90e9-92eab4c0013e None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:11.043413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa2620-7448-49f3-90e9-92eab4c0013e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:11.044890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa2620-7448-49f3-90e9-92eab4c0013e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:11.070168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 26/46] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Jul 19 15:16:10 2022] POST /identity/v3/projects => generated 312 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:16:12.761060 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307ef14-1a7d-4ceb-9374-0b90191f45f2 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:12.761610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307ef14-1a7d-4ceb-9374-0b90191f45f2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:12.761685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307ef14-1a7d-4ceb-9374-0b90191f45f2 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:12.763023 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 21/47] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:12.768260 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997640b8-3213-4e90-bd28-8ed9df2133fa None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:12.768660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997640b8-3213-4e90-bd28-8ed9df2133fa None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:12.768824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997640b8-3213-4e90-bd28-8ed9df2133fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:12.786088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-997640b8-3213-4e90-bd28-8ed9df2133fa None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:12.787269 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-997640b8-3213-4e90-bd28-8ed9df2133fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:12.855456 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 27/48] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:12.878511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307ef14-1a7d-4ceb-9374-0b90191f45f2 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:12.983704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307ef14-1a7d-4ceb-9374-0b90191f45f2 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:13.004330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:13.019421 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da08d6-02de-4b63-8641-f73bb1b62553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:13.020160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da08d6-02de-4b63-8641-f73bb1b62553 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:13.020451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da08d6-02de-4b63-8641-f73bb1b62553 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:13.020735 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da08d6-02de-4b63-8641-f73bb1b62553 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:13.031866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da08d6-02de-4b63-8641-f73bb1b62553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-c8da08d6-02de-4b63-8641-f73bb1b62553 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:13.039778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:13.040668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:13.041471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:13.041471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:13.041471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:13.041471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:13.041471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:13.041471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:13.041471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:13.041471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:13.041471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:13.041471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:13.041471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:13.041471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:13.041471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 22/49] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:16:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:13.046843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997640b8-3213-4e90-bd28-8ed9df2133fa None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:13.077337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:13.102239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba0c7a-1063-4eb2-85e5-f75fe96dd9ca None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:13.102530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba0c7a-1063-4eb2-85e5-f75fe96dd9ca None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:13.102818 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba0c7a-1063-4eb2-85e5-f75fe96dd9ca None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:13.104266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 23/50] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:13.110035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c6acc-74dc-469c-8137-e847ec5518e6 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:13.110324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c6acc-74dc-469c-8137-e847ec5518e6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:13.110606 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c6acc-74dc-469c-8137-e847ec5518e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:13.128773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-c63c6acc-74dc-469c-8137-e847ec5518e6 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:13.130404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c63c6acc-74dc-469c-8137-e847ec5518e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:13.204273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 24/51] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:13.227726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c6acc-74dc-469c-8137-e847ec5518e6 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:13.254780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997640b8-3213-4e90-bd28-8ed9df2133fa None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:13.290466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6a9bc-63da-47ba-bbfd-973b05177733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:13.291061 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6a9bc-63da-47ba-bbfd-973b05177733 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:13.291351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6a9bc-63da-47ba-bbfd-973b05177733 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:13.291636 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6a9bc-63da-47ba-bbfd-973b05177733 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:13.292401 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6a9bc-63da-47ba-bbfd-973b05177733 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:13.292711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6a9bc-63da-47ba-bbfd-973b05177733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:13.294136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6a9bc-63da-47ba-bbfd-973b05177733 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:13.306312 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 28/52] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:13.311881 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6a9bc-63da-47ba-bbfd-973b05177733 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:13.334100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c6acc-74dc-469c-8137-e847ec5518e6 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:13.345285 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:13.368947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c019137-3500-4de5-b3ab-33ec1a61c997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:13.369775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c019137-3500-4de5-b3ab-33ec1a61c997 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:13.370062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c019137-3500-4de5-b3ab-33ec1a61c997 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:13.370342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c019137-3500-4de5-b3ab-33ec1a61c997 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:13.371518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c019137-3500-4de5-b3ab-33ec1a61c997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:13.372977 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c019137-3500-4de5-b3ab-33ec1a61c997 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:13.374620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 25/53] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:13.380178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c019137-3500-4de5-b3ab-33ec1a61c997 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:13.404705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:13.412678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6a9bc-63da-47ba-bbfd-973b05177733 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:13.447492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f7343-a41d-43a5-bc54-9ab47bd0b512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:13.448367 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f7343-a41d-43a5-bc54-9ab47bd0b512 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:13.448651 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f7343-a41d-43a5-bc54-9ab47bd0b512 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:13.448932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f7343-a41d-43a5-bc54-9ab47bd0b512 None admin] PATH_INFO: `/v3/users/669959f6d82b46b1b551b6c19bf21c50` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:13.450387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f7343-a41d-43a5-bc54-9ab47bd0b512 None admin] RBAC: Authorizing `identity:get_user(user_id=669959f6d82b46b1b551b6c19bf21c50)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:13.451825 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f7343-a41d-43a5-bc54-9ab47bd0b512 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:13.453490 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 29/54] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:13 2022] GET /identity/v3/users/669959f6d82b46b1b551b6c19bf21c50 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:13.458908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f7343-a41d-43a5-bc54-9ab47bd0b512 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:13.485029 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c019137-3500-4de5-b3ab-33ec1a61c997 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:13.518935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c2a84-ec46-4cdb-8b07-07b57e807f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:13.519613 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c2a84-ec46-4cdb-8b07-07b57e807f3a None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:13.519899 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c2a84-ec46-4cdb-8b07-07b57e807f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:13.520177 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c2a84-ec46-4cdb-8b07-07b57e807f3a None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:13.520888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c2a84-ec46-4cdb-8b07-07b57e807f3a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:13.522370 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c2a84-ec46-4cdb-8b07-07b57e807f3a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:13.548941 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 26/55] 10.222.0.42 () {60 vars in 1140 bytes} [Tue Jul 19 15:16:13 2022] POST /identity/v3/projects => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:16:13.561308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f7343-a41d-43a5-bc54-9ab47bd0b512 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:13.598651 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b4808-4841-4c65-829b-5a6fea474096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:13.599485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b4808-4841-4c65-829b-5a6fea474096 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:13.599776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b4808-4841-4c65-829b-5a6fea474096 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:13.600059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b4808-4841-4c65-829b-5a6fea474096 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:13.601393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b4808-4841-4c65-829b-5a6fea474096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:13.603173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b4808-4841-4c65-829b-5a6fea474096 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:13.604965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 30/56] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:13.611323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c2a84-ec46-4cdb-8b07-07b57e807f3a None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:13.646079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:13.722878 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c2a84-ec46-4cdb-8b07-07b57e807f3a None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:13.757828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c0875-07eb-46db-9721-a5507d359151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:13.758569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c0875-07eb-46db-9721-a5507d359151 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:13.758882 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c0875-07eb-46db-9721-a5507d359151 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:13.759202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c0875-07eb-46db-9721-a5507d359151 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:13.760101 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c0875-07eb-46db-9721-a5507d359151 None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.domain.id=default, user.id=669959f6d82b46b1b551b6c19bf21c50) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:13.760423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c0875-07eb-46db-9721-a5507d359151 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:13.761931 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c0875-07eb-46db-9721-a5507d359151 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:13.767863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 27/57] 10.222.0.42 () {56 vars in 1322 bytes} [Tue Jul 19 15:16:13 2022] GET /identity/v3/role_assignments?user.id=669959f6d82b46b1b551b6c19bf21c50&scope.domain.id=default&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:15.568926 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a34aa-da9a-4d9b-a9e0-e6542b5eb61b None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:15.569452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a34aa-da9a-4d9b-a9e0-e6542b5eb61b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:15.569452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a34aa-da9a-4d9b-a9e0-e6542b5eb61b None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:15.570975 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 31/58] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:15.576201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc52d38-0e67-4597-81b9-2c316bd70682 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:15.576586 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc52d38-0e67-4597-81b9-2c316bd70682 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:15.576765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc52d38-0e67-4597-81b9-2c316bd70682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:15.593673 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-7dc52d38-0e67-4597-81b9-2c316bd70682 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:15.594845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7dc52d38-0e67-4597-81b9-2c316bd70682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:15.658967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 28/59] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:15.683577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a34aa-da9a-4d9b-a9e0-e6542b5eb61b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:15.713709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:15.784507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a34aa-da9a-4d9b-a9e0-e6542b5eb61b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:15.817416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd15a6-aef0-4481-abca-f6c0c9058f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:15.818206 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd15a6-aef0-4481-abca-f6c0c9058f1f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:15.818496 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd15a6-aef0-4481-abca-f6c0c9058f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:15.818778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd15a6-aef0-4481-abca-f6c0c9058f1f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:15.820010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd15a6-aef0-4481-abca-f6c0c9058f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:15.821478 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd15a6-aef0-4481-abca-f6c0c9058f1f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:15.823096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 32/60] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:15.828518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc52d38-0e67-4597-81b9-2c316bd70682 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:15.853377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:15.925431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc52d38-0e67-4597-81b9-2c316bd70682 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:15.958705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2730a12-6392-4962-babb-722f001cd606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:15.959372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2730a12-6392-4962-babb-722f001cd606 None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:15.959661 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2730a12-6392-4962-babb-722f001cd606 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:15.959942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2730a12-6392-4962-babb-722f001cd606 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:15.960648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2730a12-6392-4962-babb-722f001cd606 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:15.962780 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2730a12-6392-4962-babb-722f001cd606 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:15.986305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 29/61] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:16:15 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:16:19.224760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773eb0dd-9832-4b1c-b106-840be0eda54c None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:19.225697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773eb0dd-9832-4b1c-b106-840be0eda54c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:19.226124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773eb0dd-9832-4b1c-b106-840be0eda54c None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:19.228350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 33/62] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:19.233858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bb314-0f60-4806-8733-a3feaceeef3b None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:19.234626 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bb314-0f60-4806-8733-a3feaceeef3b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:19.235072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bb314-0f60-4806-8733-a3feaceeef3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:19.254153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef6f62-795f-4db7-94c8-b4a650d35094 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:19.254601 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef6f62-795f-4db7-94c8-b4a650d35094 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:19.255020 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef6f62-795f-4db7-94c8-b4a650d35094 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:19.256536 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 34/63] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:19.265795 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a845a-c04e-4389-a597-60060f2f2d21 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:19.266284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a845a-c04e-4389-a597-60060f2f2d21 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:19.266710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a845a-c04e-4389-a597-60060f2f2d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:19.279822 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-d84bb314-0f60-4806-8733-a3feaceeef3b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:19.285695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d84bb314-0f60-4806-8733-a3feaceeef3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:19.301513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7e8a845a-c04e-4389-a597-60060f2f2d21 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:19.302959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7e8a845a-c04e-4389-a597-60060f2f2d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:19.388915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 30/64] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:19.390274 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 35/65] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:19.390749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58874b89-e5b4-4a40-a537-e0bb8584ea50 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:19.391038 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58874b89-e5b4-4a40-a537-e0bb8584ea50 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:19.391469 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58874b89-e5b4-4a40-a537-e0bb8584ea50 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:19.392763 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 31/66] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:19.398990 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6e850-256c-4eec-8687-c5ba5ffbc740 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:19.399552 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6e850-256c-4eec-8687-c5ba5ffbc740 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:19.400043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6e850-256c-4eec-8687-c5ba5ffbc740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:19.412598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58874b89-e5b4-4a40-a537-e0bb8584ea50 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:19.418530 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-92b6e850-256c-4eec-8687-c5ba5ffbc740 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:19.419962 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-92b6e850-256c-4eec-8687-c5ba5ffbc740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:19.432783 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:19.450062 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:19.516085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 36/67] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:19.517827 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6e850-256c-4eec-8687-c5ba5ffbc740 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:19.557893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58874b89-e5b4-4a40-a537-e0bb8584ea50 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:19.594989 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871582a-20e7-4f8b-aabb-85d3bc72bba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:19.595805 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871582a-20e7-4f8b-aabb-85d3bc72bba5 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:19.596088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871582a-20e7-4f8b-aabb-85d3bc72bba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:19.596363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871582a-20e7-4f8b-aabb-85d3bc72bba5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:19.608245 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871582a-20e7-4f8b-aabb-85d3bc72bba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-1871582a-20e7-4f8b-aabb-85d3bc72bba5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:19.617572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:19.618447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:19.619215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:19.619215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:19.619215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:19.619215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:19.619215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:19.619215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:19.619215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:19.619215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:19.619215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:19.619215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:19.619215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:19.619215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:19.619215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 32/68] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:16:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:16:19.619837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871582a-20e7-4f8b-aabb-85d3bc72bba5 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:19.654613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6e850-256c-4eec-8687-c5ba5ffbc740 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:19.693583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c741f3c-dc4a-4965-87b7-c323972bf463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:19.694728 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c741f3c-dc4a-4965-87b7-c323972bf463 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:19.695182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c741f3c-dc4a-4965-87b7-c323972bf463 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:19.695768 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c741f3c-dc4a-4965-87b7-c323972bf463 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:19.697583 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:19.710447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c741f3c-dc4a-4965-87b7-c323972bf463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-7c741f3c-dc4a-4965-87b7-c323972bf463 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:19.719345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:19.720418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:19.721318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:19.721318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:19.721318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:19.721318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:19.721318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:19.721318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:19.721318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:19.721318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:19.721318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:19.721318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:19.721318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:19.721318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:19.721318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 37/69] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:19.722104 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae038a2-8134-41b4-81e8-612cc2b292fa None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:19.722181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae038a2-8134-41b4-81e8-612cc2b292fa None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:19.722452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae038a2-8134-41b4-81e8-612cc2b292fa None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:19.723714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 38/70] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:19.724557 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae038a2-8134-41b4-81e8-612cc2b292fa None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:19.739617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871582a-20e7-4f8b-aabb-85d3bc72bba5 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:19.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:19.778952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafe985-b92e-431c-9649-77f910147f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:19.779665 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafe985-b92e-431c-9649-77f910147f54 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:19.779951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafe985-b92e-431c-9649-77f910147f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:19.780232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafe985-b92e-431c-9649-77f910147f54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:19.790772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafe985-b92e-431c-9649-77f910147f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-5aafe985-b92e-431c-9649-77f910147f54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:19.798396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:19.799312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:19.800522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:19.800522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:19.800522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:19.800522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:19.800522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:19.800522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:19.800522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:19.800522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:19.800522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:19.800522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:19.800522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:19.800522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:19.800522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 33/71] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:16:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:19.800522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafe985-b92e-431c-9649-77f910147f54 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:19.834324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae038a2-8134-41b4-81e8-612cc2b292fa None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:19.868995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe82b2-f4cf-4304-ac01-565d33b75b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:19.869693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe82b2-f4cf-4304-ac01-565d33b75b3f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:19.869980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe82b2-f4cf-4304-ac01-565d33b75b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:19.870262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe82b2-f4cf-4304-ac01-565d33b75b3f None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:19.871018 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe82b2-f4cf-4304-ac01-565d33b75b3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:19.871329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe82b2-f4cf-4304-ac01-565d33b75b3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:19.872721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe82b2-f4cf-4304-ac01-565d33b75b3f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:19.880868 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 39/72] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:16:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:19.885555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed289991-51cb-42de-82a1-53a1d3c2e5ef None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:19.885944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed289991-51cb-42de-82a1-53a1d3c2e5ef None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:19.886238 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed289991-51cb-42de-82a1-53a1d3c2e5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:19.900480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafe985-b92e-431c-9649-77f910147f54 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:19.902440 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ed289991-51cb-42de-82a1-53a1d3c2e5ef None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:19.903887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ed289991-51cb-42de-82a1-53a1d3c2e5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:19.928965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:19.935964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af07123-6cc9-439c-8355-2e02fb656e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:19.936709 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af07123-6cc9-439c-8355-2e02fb656e67 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:19.937090 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af07123-6cc9-439c-8355-2e02fb656e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:19.937458 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af07123-6cc9-439c-8355-2e02fb656e67 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:19.938290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af07123-6cc9-439c-8355-2e02fb656e67 None admin] RBAC: Adding query filter params (name=member) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:19.938672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af07123-6cc9-439c-8355-2e02fb656e67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:19.940084 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af07123-6cc9-439c-8355-2e02fb656e67 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:19.948419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 34/73] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:19.949675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3659902-9727-4524-8141-fdb847e5d509 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:19.949966 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3659902-9727-4524-8141-fdb847e5d509 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:19.950248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3659902-9727-4524-8141-fdb847e5d509 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:19.951506 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 35/74] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:19.952207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3659902-9727-4524-8141-fdb847e5d509 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:19.978265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 40/75] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:19.979112 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed289991-51cb-42de-82a1-53a1d3c2e5ef None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:19.999374 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:20.057210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3659902-9727-4524-8141-fdb847e5d509 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:20.087738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed289991-51cb-42de-82a1-53a1d3c2e5ef None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:20.090752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3272104-51d1-4505-8cf9-6a0f57bef96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:20.091438 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3272104-51d1-4505-8cf9-6a0f57bef96a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:20.091723 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3272104-51d1-4505-8cf9-6a0f57bef96a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:20.092003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3272104-51d1-4505-8cf9-6a0f57bef96a None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:20.092751 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3272104-51d1-4505-8cf9-6a0f57bef96a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:20.093098 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3272104-51d1-4505-8cf9-6a0f57bef96a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:20.094552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3272104-51d1-4505-8cf9-6a0f57bef96a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:20.102870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 36/76] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:20.103507 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3272104-51d1-4505-8cf9-6a0f57bef96a None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:20.125903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a379d3a-569b-4f02-aa61-9d2c26e5c930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:20.126779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a379d3a-569b-4f02-aa61-9d2c26e5c930 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:20.127065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a379d3a-569b-4f02-aa61-9d2c26e5c930 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:20.127345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a379d3a-569b-4f02-aa61-9d2c26e5c930 None admin] PATH_INFO: `/v3/users/285435c09061437080caaff56db1dced` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:20.143513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a379d3a-569b-4f02-aa61-9d2c26e5c930 None admin] RBAC: Authorizing `identity:get_user(user_id=285435c09061437080caaff56db1dced)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:20.144976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a379d3a-569b-4f02-aa61-9d2c26e5c930 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:20.146660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 41/77] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:19 2022] GET /identity/v3/users/285435c09061437080caaff56db1dced => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:20.151148 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ca8f8-b807-4e0d-8078-0ec49b8d9103 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:20.151148 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ca8f8-b807-4e0d-8078-0ec49b8d9103 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:20.151148 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ca8f8-b807-4e0d-8078-0ec49b8d9103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:20.159590 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:20.167663 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d27ca8f8-b807-4e0d-8078-0ec49b8d9103 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:20.168877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d27ca8f8-b807-4e0d-8078-0ec49b8d9103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:20.206924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3272104-51d1-4505-8cf9-6a0f57bef96a None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:20.232138 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:20.238032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 42/78] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:20.238852 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ca8f8-b807-4e0d-8078-0ec49b8d9103 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:20.241840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00100c8d-a0b0-4145-b3b0-b501de667b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:20.243401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00100c8d-a0b0-4145-b3b0-b501de667b85 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:20.243490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00100c8d-a0b0-4145-b3b0-b501de667b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:20.243778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00100c8d-a0b0-4145-b3b0-b501de667b85 None admin] PATH_INFO: `/v3/users/285435c09061437080caaff56db1dced` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:20.244995 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00100c8d-a0b0-4145-b3b0-b501de667b85 None admin] RBAC: Authorizing `identity:get_user(user_id=285435c09061437080caaff56db1dced)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:20.246499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00100c8d-a0b0-4145-b3b0-b501de667b85 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:20.248141 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 37/79] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:20 2022] GET /identity/v3/users/285435c09061437080caaff56db1dced => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:20.253479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00100c8d-a0b0-4145-b3b0-b501de667b85 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:20.345766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ca8f8-b807-4e0d-8078-0ec49b8d9103 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:20.355311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00100c8d-a0b0-4145-b3b0-b501de667b85 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:20.377441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:20.379664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17caf4-4a84-45ba-8964-982131cd2b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:20.380466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17caf4-4a84-45ba-8964-982131cd2b6e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:20.380753 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17caf4-4a84-45ba-8964-982131cd2b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:20.381040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17caf4-4a84-45ba-8964-982131cd2b6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:20.382260 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc17caf4-4a84-45ba-8964-982131cd2b6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:20.384265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc17caf4-4a84-45ba-8964-982131cd2b6e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:20.385924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 43/80] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:20.386681 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17caf4-4a84-45ba-8964-982131cd2b6e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:20.390615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a75c18-9d5f-4aaa-9cda-53dbf6c0ae2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:20.391223 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a75c18-9d5f-4aaa-9cda-53dbf6c0ae2d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:20.391514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a75c18-9d5f-4aaa-9cda-53dbf6c0ae2d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:20.391793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a75c18-9d5f-4aaa-9cda-53dbf6c0ae2d None admin] PATH_INFO: `/v3/users/669959f6d82b46b1b551b6c19bf21c50` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:20.393001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a75c18-9d5f-4aaa-9cda-53dbf6c0ae2d None admin] RBAC: Authorizing `identity:get_user(user_id=669959f6d82b46b1b551b6c19bf21c50)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:20.394476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a75c18-9d5f-4aaa-9cda-53dbf6c0ae2d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:20.396319 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 38/81] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:20 2022] GET /identity/v3/users/669959f6d82b46b1b551b6c19bf21c50 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:20.397111 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a75c18-9d5f-4aaa-9cda-53dbf6c0ae2d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:20.453022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:20.490653 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17caf4-4a84-45ba-8964-982131cd2b6e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:20.496715 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a75c18-9d5f-4aaa-9cda-53dbf6c0ae2d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:20.525438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a02a25-892b-4d4d-8793-d0ff09fd300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:20.526099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a02a25-892b-4d4d-8793-d0ff09fd300a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:20.526389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a02a25-892b-4d4d-8793-d0ff09fd300a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:20.526676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a02a25-892b-4d4d-8793-d0ff09fd300a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:20.531474 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ceaf5-8100-4e3c-aa6f-b4e3b4a82686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:20.532118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ceaf5-8100-4e3c-aa6f-b4e3b4a82686 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:20.532405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ceaf5-8100-4e3c-aa6f-b4e3b4a82686 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:20.532684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ceaf5-8100-4e3c-aa6f-b4e3b4a82686 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:20.590217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:20.593824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a02a25-892b-4d4d-8793-d0ff09fd300a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:20.595309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a02a25-892b-4d4d-8793-d0ff09fd300a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:20.600596 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ceaf5-8100-4e3c-aa6f-b4e3b4a82686 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:20.602542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ceaf5-8100-4e3c-aa6f-b4e3b4a82686 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:20.659913 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:20.694681 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 44/82] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:20.696163 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a02a25-892b-4d4d-8793-d0ff09fd300a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:20.708370 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 39/83] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:20.708370 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ceaf5-8100-4e3c-aa6f-b4e3b4a82686 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:20.793280 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a02a25-892b-4d4d-8793-d0ff09fd300a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:20.796177 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:20.806930 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ceaf5-8100-4e3c-aa6f-b4e3b4a82686 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:20.826887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab4e09-41b9-46ee-9587-b6ced07749a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:20.827684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab4e09-41b9-46ee-9587-b6ced07749a1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:20.827964 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab4e09-41b9-46ee-9587-b6ced07749a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:20.828239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab4e09-41b9-46ee-9587-b6ced07749a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:20.838628 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab4e09-41b9-46ee-9587-b6ced07749a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:20.840609 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf58fe3-90ca-46bd-8279-4bf692baf2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:20.841312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf58fe3-90ca-46bd-8279-4bf692baf2e6 None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:20.841607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf58fe3-90ca-46bd-8279-4bf692baf2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:20.841892 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf58fe3-90ca-46bd-8279-4bf692baf2e6 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:20.842617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf58fe3-90ca-46bd-8279-4bf692baf2e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:20.844083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf58fe3-90ca-46bd-8279-4bf692baf2e6 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-0bab4e09-41b9-46ee-9587-b6ced07749a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:20.846343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:20.847253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:20.847999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:20.847999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:20.847999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:20.847999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:20.847999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:20.847999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:20.847999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:20.847999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:20.847999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:20.847999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:20.847999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:20.847999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:20.847999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 45/84] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:20.848620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab4e09-41b9-46ee-9587-b6ced07749a1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:20.863071 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 40/85] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:16:20 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:16:20.863816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf58fe3-90ca-46bd-8279-4bf692baf2e6 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:20.890731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:20.975947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab4e09-41b9-46ee-9587-b6ced07749a1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:20.979560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf58fe3-90ca-46bd-8279-4bf692baf2e6 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:21.015346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023be176-6850-4efe-9e3e-28800ef5852b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:21.016015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023be176-6850-4efe-9e3e-28800ef5852b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:21.016308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023be176-6850-4efe-9e3e-28800ef5852b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:21.016592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023be176-6850-4efe-9e3e-28800ef5852b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:21.022074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb2648-cd28-4b46-934e-968e313878af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:21.022828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb2648-cd28-4b46-934e-968e313878af None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:21.023780 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb2648-cd28-4b46-934e-968e313878af None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:21.023780 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb2648-cd28-4b46-934e-968e313878af None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:21.025634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb2648-cd28-4b46-934e-968e313878af None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:21.026266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb2648-cd28-4b46-934e-968e313878af None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:21.028147 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 41/86] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:20 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:21.033265 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb2648-cd28-4b46-934e-968e313878af None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:21.048913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:21.100807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023be176-6850-4efe-9e3e-28800ef5852b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:21.102416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023be176-6850-4efe-9e3e-28800ef5852b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:21.137045 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:21.168733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb2648-cd28-4b46-934e-968e313878af None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:21.213392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502053dd-83ac-4b3f-8d1e-87731b1a5e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:21.214436 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502053dd-83ac-4b3f-8d1e-87731b1a5e2b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:21.214731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502053dd-83ac-4b3f-8d1e-87731b1a5e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:21.215343 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502053dd-83ac-4b3f-8d1e-87731b1a5e2b None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:21.216823 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502053dd-83ac-4b3f-8d1e-87731b1a5e2b None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:21.219159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502053dd-83ac-4b3f-8d1e-87731b1a5e2b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:21.221608 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 42/87] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:21 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:21.222697 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502053dd-83ac-4b3f-8d1e-87731b1a5e2b None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:21.224746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 46/88] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:21.225626 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023be176-6850-4efe-9e3e-28800ef5852b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:21.323626 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:21.344734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502053dd-83ac-4b3f-8d1e-87731b1a5e2b None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:21.368907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023be176-6850-4efe-9e3e-28800ef5852b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:21.386043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef2938-e1ef-4c33-a3f4-491f298931ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:21.387350 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef2938-e1ef-4c33-a3f4-491f298931ab None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:21.388030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef2938-e1ef-4c33-a3f4-491f298931ab None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:21.388540 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef2938-e1ef-4c33-a3f4-491f298931ab None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:21.389849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef2938-e1ef-4c33-a3f4-491f298931ab None admin] RBAC: Adding query filter params (name=member) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:21.390370 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef2938-e1ef-4c33-a3f4-491f298931ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:21.392407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef2938-e1ef-4c33-a3f4-491f298931ab None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:21.397472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:21.405279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 43/89] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:21.406570 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef2938-e1ef-4c33-a3f4-491f298931ab None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:21.408630 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a4c48-0266-4d2a-96b5-ee517623d23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:21.409618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a4c48-0266-4d2a-96b5-ee517623d23e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:21.409917 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a4c48-0266-4d2a-96b5-ee517623d23e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:21.410246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a4c48-0266-4d2a-96b5-ee517623d23e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:21.411103 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a4c48-0266-4d2a-96b5-ee517623d23e None admin] RBAC: Adding query filter params (role.id=44bd561cc5594ffd9bcc27ef6dad0fc5, scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5, user.id=285435c09061437080caaff56db1dced) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:21.411465 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a4c48-0266-4d2a-96b5-ee517623d23e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:21.412916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a4c48-0266-4d2a-96b5-ee517623d23e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:21.423800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 47/90] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:21 2022] GET /identity/v3/role_assignments?user.id=285435c09061437080caaff56db1dced&scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5&role.id=44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:21.424731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a4c48-0266-4d2a-96b5-ee517623d23e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:21.565268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef2938-e1ef-4c33-a3f4-491f298931ab None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:21.605341 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a4c48-0266-4d2a-96b5-ee517623d23e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:21.611207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c799b46-da2e-4279-ad41-1d89250d7c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:21.611878 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c799b46-da2e-4279-ad41-1d89250d7c0f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:21.612164 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c799b46-da2e-4279-ad41-1d89250d7c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:21.612442 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c799b46-da2e-4279-ad41-1d89250d7c0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:21.613344 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c799b46-da2e-4279-ad41-1d89250d7c0f None admin] RBAC: Adding query filter params (role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f, scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5, user.id=285435c09061437080caaff56db1dced) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:21.613663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c799b46-da2e-4279-ad41-1d89250d7c0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:21.615278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c799b46-da2e-4279-ad41-1d89250d7c0f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:21.621737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 44/91] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:21 2022] GET /identity/v3/role_assignments?user.id=285435c09061437080caaff56db1dced&scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5&role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:21.622621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c799b46-da2e-4279-ad41-1d89250d7c0f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:21.635325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:21.659434 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca2747-8fa2-426d-8fe5-75a78d038050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:21.660728 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca2747-8fa2-426d-8fe5-75a78d038050 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:21.661347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca2747-8fa2-426d-8fe5-75a78d038050 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:21.661862 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca2747-8fa2-426d-8fe5-75a78d038050 None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:21.663505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca2747-8fa2-426d-8fe5-75a78d038050 None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:21.666028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca2747-8fa2-426d-8fe5-75a78d038050 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:21.668392 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 48/92] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:21 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:21.674454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca2747-8fa2-426d-8fe5-75a78d038050 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:21.752766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c799b46-da2e-4279-ad41-1d89250d7c0f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:21.773788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:21.797572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925b493-012a-4ffa-80d0-fd553531ac9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:21.798468 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925b493-012a-4ffa-80d0-fd553531ac9e None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:21.798757 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925b493-012a-4ffa-80d0-fd553531ac9e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:21.799037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925b493-012a-4ffa-80d0-fd553531ac9e None admin] PATH_INFO: `/v3/users/285435c09061437080caaff56db1dced` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:21.800265 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925b493-012a-4ffa-80d0-fd553531ac9e None admin] RBAC: Authorizing `identity:get_user(user_id=285435c09061437080caaff56db1dced)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:21.801994 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925b493-012a-4ffa-80d0-fd553531ac9e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:21.804084 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 45/93] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:21 2022] GET /identity/v3/users/285435c09061437080caaff56db1dced => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:21.813202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925b493-012a-4ffa-80d0-fd553531ac9e None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:21.865893 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca2747-8fa2-426d-8fe5-75a78d038050 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:21.942571 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f9cea-c9f0-4623-8986-b3960b55aef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:21.943179 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f9cea-c9f0-4623-8986-b3960b55aef0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:21.943469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f9cea-c9f0-4623-8986-b3960b55aef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:21.943752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f9cea-c9f0-4623-8986-b3960b55aef0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:21.944618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f9cea-c9f0-4623-8986-b3960b55aef0 None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5, user.id=669959f6d82b46b1b551b6c19bf21c50) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:21.945488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f9cea-c9f0-4623-8986-b3960b55aef0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:21.947625 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f9cea-c9f0-4623-8986-b3960b55aef0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:21.958365 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:21.962764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 49/94] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:21 2022] GET /identity/v3/role_assignments?user.id=669959f6d82b46b1b551b6c19bf21c50&scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:21.987192 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925b493-012a-4ffa-80d0-fd553531ac9e None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:22.028134 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f633ce-afb9-49ef-a779-86760cf758aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:22.029478 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f633ce-afb9-49ef-a779-86760cf758aa None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:22.029976 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f633ce-afb9-49ef-a779-86760cf758aa None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:22.030505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f633ce-afb9-49ef-a779-86760cf758aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:22.126153 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f633ce-afb9-49ef-a779-86760cf758aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:22.128048 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f633ce-afb9-49ef-a779-86760cf758aa None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:22.217016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:22.245621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 46/95] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:22.255892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f9cea-c9f0-4623-8986-b3960b55aef0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:22.391234 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f9cea-c9f0-4623-8986-b3960b55aef0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:22.432929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:22.449606 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f3d7c-0657-4d7a-9a8e-b9c681bd77f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:22.450827 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f3d7c-0657-4d7a-9a8e-b9c681bd77f2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:22.451317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f3d7c-0657-4d7a-9a8e-b9c681bd77f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:22.451816 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f3d7c-0657-4d7a-9a8e-b9c681bd77f2 None admin] PATH_INFO: `/v3/projects/fc3a144aaec74055a742212b2a2c746f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:22.453281 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f3d7c-0657-4d7a-9a8e-b9c681bd77f2 None admin] RBAC: Authorizing `identity:get_project(project_id=fc3a144aaec74055a742212b2a2c746f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:22.454954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f3d7c-0657-4d7a-9a8e-b9c681bd77f2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:22.456890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 50/96] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:22 2022] GET /identity/v3/projects/fc3a144aaec74055a742212b2a2c746f => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:22.463659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f633ce-afb9-49ef-a779-86760cf758aa None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:22.592096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f633ce-afb9-49ef-a779-86760cf758aa None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:22.628560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa13b4-49f3-489f-bc47-6de5f7cf078d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:22.629623 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa13b4-49f3-489f-bc47-6de5f7cf078d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:22.630152 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa13b4-49f3-489f-bc47-6de5f7cf078d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:22.630634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa13b4-49f3-489f-bc47-6de5f7cf078d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:22.631750 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa13b4-49f3-489f-bc47-6de5f7cf078d None admin] RBAC: Adding query filter params (role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f, scope.project.id=fc3a144aaec74055a742212b2a2c746f, user.id=285435c09061437080caaff56db1dced) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:22.632269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa13b4-49f3-489f-bc47-6de5f7cf078d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:22.634213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa13b4-49f3-489f-bc47-6de5f7cf078d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:22.640167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 47/97] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:22 2022] GET /identity/v3/role_assignments?user.id=285435c09061437080caaff56db1dced&scope.project.id=fc3a144aaec74055a742212b2a2c746f&role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:23.769699 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d5e95-7d3a-4ad2-b34f-7c7740f4613d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:23.770205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d5e95-7d3a-4ad2-b34f-7c7740f4613d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:23.770278 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d5e95-7d3a-4ad2-b34f-7c7740f4613d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:23.771661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 51/98] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:23.777724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718943c1-1a24-49e3-9cc9-b3eb214d0fd4 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:23.778130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718943c1-1a24-49e3-9cc9-b3eb214d0fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:23.778590 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718943c1-1a24-49e3-9cc9-b3eb214d0fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:23.799970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-718943c1-1a24-49e3-9cc9-b3eb214d0fd4 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:23.801229 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-718943c1-1a24-49e3-9cc9-b3eb214d0fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:23.852662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:23.897202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 48/99] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:23.942673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d5e95-7d3a-4ad2-b34f-7c7740f4613d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:24.090775 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d5e95-7d3a-4ad2-b34f-7c7740f4613d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:24.139210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5423090-b5dd-4a98-8d82-77189fcf8473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:24.140375 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5423090-b5dd-4a98-8d82-77189fcf8473 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:24.140879 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5423090-b5dd-4a98-8d82-77189fcf8473 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:24.141369 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5423090-b5dd-4a98-8d82-77189fcf8473 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:24.155347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5423090-b5dd-4a98-8d82-77189fcf8473 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:24.164555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d2a6f-91bc-4e77-abd9-bbef692736ff None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:24.164865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d2a6f-91bc-4e77-abd9-bbef692736ff None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:24.165178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d2a6f-91bc-4e77-abd9-bbef692736ff None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e5423090-b5dd-4a98-8d82-77189fcf8473 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:24.166047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:24.166865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:24.167696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:24.167696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:24.167696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:24.167696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:24.167696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:24.167696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:24.167696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:24.167696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:24.167696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:24.167696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:24.167696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:24.167696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:24.168268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 49/100] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:24.170304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 52/101] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:24.175692 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cb84d-84ca-46b3-a2dd-e16fcb6c7586 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:24.175990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cb84d-84ca-46b3-a2dd-e16fcb6c7586 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:24.176275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cb84d-84ca-46b3-a2dd-e16fcb6c7586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:24.176894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5423090-b5dd-4a98-8d82-77189fcf8473 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:24.200388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-8b1cb84d-84ca-46b3-a2dd-e16fcb6c7586 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:24.209726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-8b1cb84d-84ca-46b3-a2dd-e16fcb6c7586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:24.278900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:24.336982 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5423090-b5dd-4a98-8d82-77189fcf8473 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:24.352899 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 50/102] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:24.354297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616dc44-8731-43c9-bbde-ba15f909ecec None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:24.354596 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616dc44-8731-43c9-bbde-ba15f909ecec None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:24.354882 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616dc44-8731-43c9-bbde-ba15f909ecec None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:24.356247 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 51/103] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:24.370464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619eaf5d-0e87-4601-b674-94fbb3170119 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:24.370758 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619eaf5d-0e87-4601-b674-94fbb3170119 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:24.371044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619eaf5d-0e87-4601-b674-94fbb3170119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:24.380528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f8308-9357-400e-be2a-24f8fc5ca426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:24.381321 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f8308-9357-400e-be2a-24f8fc5ca426 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:24.381708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f8308-9357-400e-be2a-24f8fc5ca426 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:24.381998 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f8308-9357-400e-be2a-24f8fc5ca426 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:24.383416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f8308-9357-400e-be2a-24f8fc5ca426 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:24.383733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f8308-9357-400e-be2a-24f8fc5ca426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:24.385233 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f8308-9357-400e-be2a-24f8fc5ca426 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:24.394266 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 53/104] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:24.400157 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f8308-9357-400e-be2a-24f8fc5ca426 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:24.425035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-619eaf5d-0e87-4601-b674-94fbb3170119 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:24.426267 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-619eaf5d-0e87-4601-b674-94fbb3170119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:24.561131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f8308-9357-400e-be2a-24f8fc5ca426 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:24.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 52/105] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:24.564678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619eaf5d-0e87-4601-b674-94fbb3170119 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:24.584854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:24.620570 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:24.626521 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f81f2-2d4c-4d5a-8417-e4dd9230062a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:24.627340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f81f2-2d4c-4d5a-8417-e4dd9230062a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:24.627633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f81f2-2d4c-4d5a-8417-e4dd9230062a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:24.627917 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f81f2-2d4c-4d5a-8417-e4dd9230062a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:24.633947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f81f2-2d4c-4d5a-8417-e4dd9230062a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:24.635426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f81f2-2d4c-4d5a-8417-e4dd9230062a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:24.637093 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 54/106] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:24.638492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f81f2-2d4c-4d5a-8417-e4dd9230062a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:24.726443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619eaf5d-0e87-4601-b674-94fbb3170119 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:24.767438 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0ba3f-f2bd-473f-9444-33b2086b0c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:24.768356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0ba3f-f2bd-473f-9444-33b2086b0c2f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:24.768645 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0ba3f-f2bd-473f-9444-33b2086b0c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:24.768924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0ba3f-f2bd-473f-9444-33b2086b0c2f None admin] PATH_INFO: `/v3/users/581c8d476f744f25ad957b35c68a5fe2` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:24.791364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0ba3f-f2bd-473f-9444-33b2086b0c2f None admin] RBAC: Authorizing `identity:get_user(user_id=581c8d476f744f25ad957b35c68a5fe2)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:24.792911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0ba3f-f2bd-473f-9444-33b2086b0c2f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:24.794897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 53/107] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:24 2022] GET /identity/v3/users/581c8d476f744f25ad957b35c68a5fe2 => generated 289 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:24.795649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0ba3f-f2bd-473f-9444-33b2086b0c2f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:24.803545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f81f2-2d4c-4d5a-8417-e4dd9230062a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:24.840583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db0e70-f2fe-4d34-aa43-0b70460108c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:24.841716 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db0e70-f2fe-4d34-aa43-0b70460108c8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:24.842144 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db0e70-f2fe-4d34-aa43-0b70460108c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:24.842563 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db0e70-f2fe-4d34-aa43-0b70460108c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:24.854711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db0e70-f2fe-4d34-aa43-0b70460108c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c7db0e70-f2fe-4d34-aa43-0b70460108c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:24.865201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:24.866258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:24.867991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:24.867991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:24.867991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:24.867991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:24.867991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:24.867991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:24.867991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:24.867991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:24.867991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:24.867991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:24.867991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:24.867991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:24.870478 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 55/108] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:16:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:24.872515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce069c5-ba6c-4cbf-a7a3-4fbf3f8ccc55 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:24.872953 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce069c5-ba6c-4cbf-a7a3-4fbf3f8ccc55 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:24.873396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce069c5-ba6c-4cbf-a7a3-4fbf3f8ccc55 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:24.875690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 56/109] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:24.876848 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce069c5-ba6c-4cbf-a7a3-4fbf3f8ccc55 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:24.900960 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:24.918613 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0ba3f-f2bd-473f-9444-33b2086b0c2f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:24.937776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:24.961781 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93005c0c-097e-4679-b088-7a74d72ca8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:24.963476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93005c0c-097e-4679-b088-7a74d72ca8bb None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:24.963965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93005c0c-097e-4679-b088-7a74d72ca8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:24.964684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93005c0c-097e-4679-b088-7a74d72ca8bb None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:24.966009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93005c0c-097e-4679-b088-7a74d72ca8bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:24.967846 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93005c0c-097e-4679-b088-7a74d72ca8bb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:25.004201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 54/110] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Jul 19 15:16:24 2022] POST /identity/v3/projects => generated 302 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:16:25.005933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93005c0c-097e-4679-b088-7a74d72ca8bb None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:25.030716 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce069c5-ba6c-4cbf-a7a3-4fbf3f8ccc55 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:25.119060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87606f-ccef-4f2e-8c4f-b633c1cefaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:25.119750 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87606f-ccef-4f2e-8c4f-b633c1cefaef None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:25.120036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87606f-ccef-4f2e-8c4f-b633c1cefaef None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:25.120313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87606f-ccef-4f2e-8c4f-b633c1cefaef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:25.163518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93005c0c-097e-4679-b088-7a74d72ca8bb None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:25.208072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74726788-8683-4fe2-b60c-6196df2d40fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:25.209224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74726788-8683-4fe2-b60c-6196df2d40fb None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:25.209821 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74726788-8683-4fe2-b60c-6196df2d40fb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:25.210198 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74726788-8683-4fe2-b60c-6196df2d40fb None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:25.212324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74726788-8683-4fe2-b60c-6196df2d40fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:25.212950 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74726788-8683-4fe2-b60c-6196df2d40fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:25.214607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74726788-8683-4fe2-b60c-6196df2d40fb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:25.223080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87606f-ccef-4f2e-8c4f-b633c1cefaef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:25.224578 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87606f-ccef-4f2e-8c4f-b633c1cefaef None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:25.225509 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 55/111] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:16:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:25.228384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34715265-9e1d-417c-a06a-d9f136f10134 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:25.228951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34715265-9e1d-417c-a06a-d9f136f10134 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:25.229443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34715265-9e1d-417c-a06a-d9f136f10134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:25.234773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:25.255263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-34715265-9e1d-417c-a06a-d9f136f10134 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:25.256489 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-34715265-9e1d-417c-a06a-d9f136f10134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:25.291722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:25.344539 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 56/112] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:25.346036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34715265-9e1d-417c-a06a-d9f136f10134 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:25.380995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 57/113] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:25.383170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa4303-c366-41ea-8c33-37676c7b0db5 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:25.383718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa4303-c366-41ea-8c33-37676c7b0db5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:25.384231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa4303-c366-41ea-8c33-37676c7b0db5 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:25.386267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 58/114] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:25.387244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa4303-c366-41ea-8c33-37676c7b0db5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:25.477559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34715265-9e1d-417c-a06a-d9f136f10134 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:25.501267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa4303-c366-41ea-8c33-37676c7b0db5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:25.508581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:25.514998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5dc89-2b2e-4e62-8069-4be5635d3624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:25.515884 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5dc89-2b2e-4e62-8069-4be5635d3624 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:25.516167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5dc89-2b2e-4e62-8069-4be5635d3624 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:25.516451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5dc89-2b2e-4e62-8069-4be5635d3624 None admin] PATH_INFO: `/v3/users/285435c09061437080caaff56db1dced` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:25.517674 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad5dc89-2b2e-4e62-8069-4be5635d3624 None admin] RBAC: Authorizing `identity:get_user(user_id=285435c09061437080caaff56db1dced)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:25.519124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad5dc89-2b2e-4e62-8069-4be5635d3624 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:25.520931 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 57/115] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:25 2022] GET /identity/v3/users/285435c09061437080caaff56db1dced => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:25.521644 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5dc89-2b2e-4e62-8069-4be5635d3624 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:25.541566 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3181b21-71be-474f-babb-c0b61a845443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:25.542420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3181b21-71be-474f-babb-c0b61a845443 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:25.542648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3181b21-71be-474f-babb-c0b61a845443 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:25.542945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3181b21-71be-474f-babb-c0b61a845443 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:25.551355 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:25.555092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3181b21-71be-474f-babb-c0b61a845443 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d3181b21-71be-474f-babb-c0b61a845443 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:25.563231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:25.564150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:25.564925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:25.564925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:25.564925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:25.564925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:25.564925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:25.564925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:25.564925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:25.564925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:25.564925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:25.564925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:25.564925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:25.564925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:25.564925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 59/116] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:25.567028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efcde0-f172-4794-acd1-d650448325d9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:25.567028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efcde0-f172-4794-acd1-d650448325d9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:25.567028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efcde0-f172-4794-acd1-d650448325d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:25.583111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-76efcde0-f172-4794-acd1-d650448325d9 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:25.584098 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-76efcde0-f172-4794-acd1-d650448325d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:25.646503 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5dc89-2b2e-4e62-8069-4be5635d3624 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:25.662199 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 60/117] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:25.663794 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c24b0-756c-4ebb-b980-b43bab62e930 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:25.664286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c24b0-756c-4ebb-b980-b43bab62e930 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:25.664781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c24b0-756c-4ebb-b980-b43bab62e930 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:25.666317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 61/118] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:25.667166 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c24b0-756c-4ebb-b980-b43bab62e930 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:25.695413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e6f96-5919-438a-b1bd-8f3c877ada82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:25.696451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e6f96-5919-438a-b1bd-8f3c877ada82 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:25.696774 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e6f96-5919-438a-b1bd-8f3c877ada82 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:25.697057 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e6f96-5919-438a-b1bd-8f3c877ada82 None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:25.698300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e6f96-5919-438a-b1bd-8f3c877ada82 None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:25.699775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e6f96-5919-438a-b1bd-8f3c877ada82 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:25.702667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 58/119] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:25 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:25.702667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e6f96-5919-438a-b1bd-8f3c877ada82 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:25.777490 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c24b0-756c-4ebb-b980-b43bab62e930 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:25.781585 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:25.807621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:25.814277 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6be93-46b7-4ec5-a4f5-e88dd5fd3cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:25.815145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6be93-46b7-4ec5-a4f5-e88dd5fd3cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:25.815532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6be93-46b7-4ec5-a4f5-e88dd5fd3cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:25.815793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6be93-46b7-4ec5-a4f5-e88dd5fd3cd5 None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:25.816891 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6be93-46b7-4ec5-a4f5-e88dd5fd3cd5 None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:25.817673 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e6f96-5919-438a-b1bd-8f3c877ada82 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:25.818380 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6be93-46b7-4ec5-a4f5-e88dd5fd3cd5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:25.821176 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 62/120] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:25 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:25.822138 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5b63e-4a22-4187-aeb8-339a98b2a9dc None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:25.822426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5b63e-4a22-4187-aeb8-339a98b2a9dc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:25.822703 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5b63e-4a22-4187-aeb8-339a98b2a9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:25.840196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-20a5b63e-4a22-4187-aeb8-339a98b2a9dc None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:25.841400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-20a5b63e-4a22-4187-aeb8-339a98b2a9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:25.860151 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a62e94-4bd0-4e87-9f84-7c8f199c937b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:25.860834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a62e94-4bd0-4e87-9f84-7c8f199c937b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:25.861128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a62e94-4bd0-4e87-9f84-7c8f199c937b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:25.861437 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a62e94-4bd0-4e87-9f84-7c8f199c937b None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:25.862247 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a62e94-4bd0-4e87-9f84-7c8f199c937b None admin] RBAC: Adding query filter params (name=member) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:25.862518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a62e94-4bd0-4e87-9f84-7c8f199c937b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:25.863915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a62e94-4bd0-4e87-9f84-7c8f199c937b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:25.872353 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 59/121] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:25.875313 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a62e94-4bd0-4e87-9f84-7c8f199c937b None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:25.915886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 63/122] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:25.916528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5b63e-4a22-4187-aeb8-339a98b2a9dc None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:25.995622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a62e94-4bd0-4e87-9f84-7c8f199c937b None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:26.024706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:26.025818 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5b63e-4a22-4187-aeb8-339a98b2a9dc None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:26.032073 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018b6be-a14f-48a0-85e9-2f243c9eaf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:26.032809 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018b6be-a14f-48a0-85e9-2f243c9eaf3f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:26.033099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018b6be-a14f-48a0-85e9-2f243c9eaf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:26.033400 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018b6be-a14f-48a0-85e9-2f243c9eaf3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:26.044537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018b6be-a14f-48a0-85e9-2f243c9eaf3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-f018b6be-a14f-48a0-85e9-2f243c9eaf3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:26.052698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:26.053740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:26.055314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:26.055314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:26.055314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:26.055314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:26.055314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:26.055314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:26.055314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:26.055314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:26.055314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:26.055314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:26.055314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:26.055314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:26.055314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 60/123] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:16:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:26.055314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018b6be-a14f-48a0-85e9-2f243c9eaf3f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:26.062664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:26.072784 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3a201-e69d-4fbc-abc2-ecc26047ada8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:26.073793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3a201-e69d-4fbc-abc2-ecc26047ada8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:26.074221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3a201-e69d-4fbc-abc2-ecc26047ada8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:26.074787 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3a201-e69d-4fbc-abc2-ecc26047ada8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:26.075764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e3a201-e69d-4fbc-abc2-ecc26047ada8 None admin] RBAC: Adding query filter params (role.id=64202be313d04a5aab01b866bc558152, scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5, user.id=581c8d476f744f25ad957b35c68a5fe2) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:26.076240 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e3a201-e69d-4fbc-abc2-ecc26047ada8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:26.078103 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e3a201-e69d-4fbc-abc2-ecc26047ada8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:26.084202 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 64/124] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:25 2022] GET /identity/v3/role_assignments?user.id=581c8d476f744f25ad957b35c68a5fe2&scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5&role.id=64202be313d04a5aab01b866bc558152 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:26.085125 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3a201-e69d-4fbc-abc2-ecc26047ada8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:26.185345 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018b6be-a14f-48a0-85e9-2f243c9eaf3f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:26.227193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fcd60-84ec-4858-b712-2be2bdd0e2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:26.228119 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fcd60-84ec-4858-b712-2be2bdd0e2fb None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:26.228409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fcd60-84ec-4858-b712-2be2bdd0e2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:26.228704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fcd60-84ec-4858-b712-2be2bdd0e2fb None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5/users/285435c09061437080caaff56db1dced/roles/44bd561cc5594ffd9bcc27ef6dad0fc5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:26.231115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fcd60-84ec-4858-b712-2be2bdd0e2fb None admin] RBAC: Authorizing `identity:create_grant(project_id=30d54bfc38f04330a2dc73fcb0219ec5, user_id=285435c09061437080caaff56db1dced, role_id=44bd561cc5594ffd9bcc27ef6dad0fc5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:26.232470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3a201-e69d-4fbc-abc2-ecc26047ada8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:26.232618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fcd60-84ec-4858-b712-2be2bdd0e2fb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:26.253731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 61/125] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:16:26 2022] PUT /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5/users/285435c09061437080caaff56db1dced/roles/44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:16:26.254839 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fcd60-84ec-4858-b712-2be2bdd0e2fb None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:26.280206 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcf074-7dd7-439e-991b-2f71d13598c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:26.281746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcf074-7dd7-439e-991b-2f71d13598c0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:26.282231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcf074-7dd7-439e-991b-2f71d13598c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:26.282749 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcf074-7dd7-439e-991b-2f71d13598c0 None admin] PATH_INFO: `/v3/users/285435c09061437080caaff56db1dced` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:26.284221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcf074-7dd7-439e-991b-2f71d13598c0 None admin] RBAC: Authorizing `identity:get_user(user_id=285435c09061437080caaff56db1dced)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:26.285891 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcf074-7dd7-439e-991b-2f71d13598c0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:26.289389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 65/126] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:26 2022] GET /identity/v3/users/285435c09061437080caaff56db1dced => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:26.290819 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcf074-7dd7-439e-991b-2f71d13598c0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:26.318938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:26.364379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:26.387191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fcd60-84ec-4858-b712-2be2bdd0e2fb None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:26.435102 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d4c97-ef25-4ebe-a636-c6f5c0458e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:26.435934 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d4c97-ef25-4ebe-a636-c6f5c0458e98 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:26.436225 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d4c97-ef25-4ebe-a636-c6f5c0458e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:26.436511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d4c97-ef25-4ebe-a636-c6f5c0458e98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:26.438689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcf074-7dd7-439e-991b-2f71d13598c0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:26.461790 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d4c97-ef25-4ebe-a636-c6f5c0458e98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-070d4c97-ef25-4ebe-a636-c6f5c0458e98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:26.470390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:26.471295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:26.472490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:26.472490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:26.472490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:26.472490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:26.472490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:26.472490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:26.472490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:26.472490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:26.472490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:26.472490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:26.472490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:26.472490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:26.472490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 62/127] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:26.473446 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d4c97-ef25-4ebe-a636-c6f5c0458e98 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:26.486987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3143f7-45aa-4d82-8e33-365ee2993ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:26.487679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3143f7-45aa-4d82-8e33-365ee2993ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:26.488072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3143f7-45aa-4d82-8e33-365ee2993ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:26.488363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3143f7-45aa-4d82-8e33-365ee2993ef0 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:26.489151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3143f7-45aa-4d82-8e33-365ee2993ef0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:26.489579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3143f7-45aa-4d82-8e33-365ee2993ef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:26.490982 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3143f7-45aa-4d82-8e33-365ee2993ef0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:26.500810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 66/128] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:26.501680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3143f7-45aa-4d82-8e33-365ee2993ef0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:26.595126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d4c97-ef25-4ebe-a636-c6f5c0458e98 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:26.607451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:26.613658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3143f7-45aa-4d82-8e33-365ee2993ef0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:26.634689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca9310-ad72-4da2-a81f-7b5a7f71667d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:26.635664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca9310-ad72-4da2-a81f-7b5a7f71667d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:26.636061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca9310-ad72-4da2-a81f-7b5a7f71667d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:26.636353 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca9310-ad72-4da2-a81f-7b5a7f71667d None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:26.637619 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca9310-ad72-4da2-a81f-7b5a7f71667d None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:26.638873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:26.640287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca9310-ad72-4da2-a81f-7b5a7f71667d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:26.642647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 63/129] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:26 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:26.643901 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca9310-ad72-4da2-a81f-7b5a7f71667d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:26.662888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc554d6b-3759-49ea-8606-4b24fa5066f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:26.664139 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc554d6b-3759-49ea-8606-4b24fa5066f6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:26.664429 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc554d6b-3759-49ea-8606-4b24fa5066f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:26.664901 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc554d6b-3759-49ea-8606-4b24fa5066f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:26.665809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc554d6b-3759-49ea-8606-4b24fa5066f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:26.666131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc554d6b-3759-49ea-8606-4b24fa5066f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:26.667629 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc554d6b-3759-49ea-8606-4b24fa5066f6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:26.685757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 67/130] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:26.686588 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc554d6b-3759-49ea-8606-4b24fa5066f6 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:26.754461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca9310-ad72-4da2-a81f-7b5a7f71667d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:26.793125 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554296e-4a3d-44bf-9daa-fee56d43ede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:26.794321 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554296e-4a3d-44bf-9daa-fee56d43ede2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:26.794866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554296e-4a3d-44bf-9daa-fee56d43ede2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:26.795298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554296e-4a3d-44bf-9daa-fee56d43ede2 None admin] PATH_INFO: `/v3/users/669959f6d82b46b1b551b6c19bf21c50` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:26.796801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554296e-4a3d-44bf-9daa-fee56d43ede2 None admin] RBAC: Authorizing `identity:get_user(user_id=669959f6d82b46b1b551b6c19bf21c50)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:26.798505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554296e-4a3d-44bf-9daa-fee56d43ede2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:26.800333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 64/131] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:26 2022] GET /identity/v3/users/669959f6d82b46b1b551b6c19bf21c50 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:26.801275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554296e-4a3d-44bf-9daa-fee56d43ede2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:26.861642 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:26.882979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc554d6b-3759-49ea-8606-4b24fa5066f6 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:26.914041 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554296e-4a3d-44bf-9daa-fee56d43ede2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:26.925983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f9af7-ba1a-44e0-b564-4e483685b981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:26.926942 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f9af7-ba1a-44e0-b564-4e483685b981 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:26.927185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f9af7-ba1a-44e0-b564-4e483685b981 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:26.927519 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f9af7-ba1a-44e0-b564-4e483685b981 None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5/users/285435c09061437080caaff56db1dced/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:26.930305 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f9af7-ba1a-44e0-b564-4e483685b981 None admin] RBAC: Authorizing `identity:create_grant(project_id=30d54bfc38f04330a2dc73fcb0219ec5, user_id=285435c09061437080caaff56db1dced, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:26.931788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f9af7-ba1a-44e0-b564-4e483685b981 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:26.954782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35521f-6250-4777-9584-60cf89c498b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:26.956203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35521f-6250-4777-9584-60cf89c498b5 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:26.956703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35521f-6250-4777-9584-60cf89c498b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:26.957242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35521f-6250-4777-9584-60cf89c498b5 None admin] PATH_INFO: `/v3/users/285435c09061437080caaff56db1dced` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:26.958725 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35521f-6250-4777-9584-60cf89c498b5 None admin] RBAC: Authorizing `identity:get_user(user_id=285435c09061437080caaff56db1dced)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:26.960543 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35521f-6250-4777-9584-60cf89c498b5 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:26.962762 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 65/132] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:26 2022] GET /identity/v3/users/285435c09061437080caaff56db1dced => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:26.963814 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35521f-6250-4777-9584-60cf89c498b5 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:26.965455 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 68/133] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:16:26 2022] PUT /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5/users/285435c09061437080caaff56db1dced/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:16:26.972080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f9af7-ba1a-44e0-b564-4e483685b981 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:27.057379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:27.087112 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35521f-6250-4777-9584-60cf89c498b5 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:27.104221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:27.128628 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4964b5a-9967-47a9-8cbc-a2b1f9cfd0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:27.129878 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964b5a-9967-47a9-8cbc-a2b1f9cfd0bf None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:27.130383 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964b5a-9967-47a9-8cbc-a2b1f9cfd0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:27.130872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964b5a-9967-47a9-8cbc-a2b1f9cfd0bf None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:27.133015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4964b5a-9967-47a9-8cbc-a2b1f9cfd0bf None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:27.134776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4964b5a-9967-47a9-8cbc-a2b1f9cfd0bf None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:27.136933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 66/134] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:26 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:27.139916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f9af7-ba1a-44e0-b564-4e483685b981 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:27.146174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4964b5a-9967-47a9-8cbc-a2b1f9cfd0bf None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:27.209629 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beefed5-e8a8-4c9a-b153-dc9510b12810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:27.211078 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beefed5-e8a8-4c9a-b153-dc9510b12810 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:27.211367 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beefed5-e8a8-4c9a-b153-dc9510b12810 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:27.211648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beefed5-e8a8-4c9a-b153-dc9510b12810 None admin] PATH_INFO: `/v3/projects/fc3a144aaec74055a742212b2a2c746f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:27.213111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beefed5-e8a8-4c9a-b153-dc9510b12810 None admin] RBAC: Authorizing `identity:get_project(project_id=fc3a144aaec74055a742212b2a2c746f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:27.214661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beefed5-e8a8-4c9a-b153-dc9510b12810 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:27.220110 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 69/135] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:26 2022] GET /identity/v3/projects/fc3a144aaec74055a742212b2a2c746f => generated 312 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:27.226211 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beefed5-e8a8-4c9a-b153-dc9510b12810 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:27.310380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4964b5a-9967-47a9-8cbc-a2b1f9cfd0bf None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:27.356856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a56c0-e2b3-4d55-94a1-bd0de8403d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:27.358038 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a56c0-e2b3-4d55-94a1-bd0de8403d17 None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:27.358530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a56c0-e2b3-4d55-94a1-bd0de8403d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:27.358966 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a56c0-e2b3-4d55-94a1-bd0de8403d17 None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5/users/669959f6d82b46b1b551b6c19bf21c50/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:27.363346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a56c0-e2b3-4d55-94a1-bd0de8403d17 None admin] RBAC: Authorizing `identity:create_grant(project_id=30d54bfc38f04330a2dc73fcb0219ec5, user_id=669959f6d82b46b1b551b6c19bf21c50, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:27.368204 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a56c0-e2b3-4d55-94a1-bd0de8403d17 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:27.393998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 67/136] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:16:27 2022] PUT /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5/users/669959f6d82b46b1b551b6c19bf21c50/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:16:27.443648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea5018-a95d-4638-812b-ee5c5d9c3669 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:27.443942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea5018-a95d-4638-812b-ee5c5d9c3669 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:27.444239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea5018-a95d-4638-812b-ee5c5d9c3669 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:27.446205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 68/137] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:16:27.450586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beefed5-e8a8-4c9a-b153-dc9510b12810 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:27.451822 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a39bd8-6789-49d3-8f14-4ea9b7892750 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:27.452115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a39bd8-6789-49d3-8f14-4ea9b7892750 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:27.452400 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a39bd8-6789-49d3-8f14-4ea9b7892750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:27.470010 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-74a39bd8-6789-49d3-8f14-4ea9b7892750 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:27.471323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-74a39bd8-6789-49d3-8f14-4ea9b7892750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:27.481080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:27.511662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:27.548928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 69/138] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:27.555017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149bc60-bcc9-42d1-b8ec-d4496719cb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:27.556291 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149bc60-bcc9-42d1-b8ec-d4496719cb50 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:27.556745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149bc60-bcc9-42d1-b8ec-d4496719cb50 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:27.557207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149bc60-bcc9-42d1-b8ec-d4496719cb50 None admin] PATH_INFO: `/v3/projects/fc3a144aaec74055a742212b2a2c746f/users/285435c09061437080caaff56db1dced/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:27.562530 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149bc60-bcc9-42d1-b8ec-d4496719cb50 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc3a144aaec74055a742212b2a2c746f, user_id=285435c09061437080caaff56db1dced, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:27.564205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149bc60-bcc9-42d1-b8ec-d4496719cb50 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:27.573881 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a39bd8-6789-49d3-8f14-4ea9b7892750 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:27.600890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 70/139] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:16:27 2022] PUT /identity/v3/projects/fc3a144aaec74055a742212b2a2c746f/users/285435c09061437080caaff56db1dced/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:16:27.744325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a39bd8-6789-49d3-8f14-4ea9b7892750 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:27.785708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6643415-60f3-4e6f-858d-955d8a00dd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:27.786512 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6643415-60f3-4e6f-858d-955d8a00dd3d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:27.786793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6643415-60f3-4e6f-858d-955d8a00dd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:27.787069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6643415-60f3-4e6f-858d-955d8a00dd3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:27.788245 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6643415-60f3-4e6f-858d-955d8a00dd3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:27.790228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6643415-60f3-4e6f-858d-955d8a00dd3d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:27.791888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 70/140] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:27.797366 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149bc60-bcc9-42d1-b8ec-d4496719cb50 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:27.941378 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149bc60-bcc9-42d1-b8ec-d4496719cb50 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:27.999488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c541bb9-18ce-499d-88c2-43497ba67ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:28.000103 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c541bb9-18ce-499d-88c2-43497ba67ca1 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:28.000389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c541bb9-18ce-499d-88c2-43497ba67ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:28.000665 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c541bb9-18ce-499d-88c2-43497ba67ca1 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:28.001445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c541bb9-18ce-499d-88c2-43497ba67ca1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:28.003041 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c541bb9-18ce-499d-88c2-43497ba67ca1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:28.040424 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 71/141] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:16:27 2022] POST /identity/v3/users => generated 283 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:16:29.042058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8962e0-4071-4679-9d32-7745d74332f6 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:29.042566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8962e0-4071-4679-9d32-7745d74332f6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:29.042672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8962e0-4071-4679-9d32-7745d74332f6 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:29.044251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 71/142] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:29.050824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd101af-e9c3-457f-b793-5caaffc39884 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:29.051269 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd101af-e9c3-457f-b793-5caaffc39884 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:29.051444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd101af-e9c3-457f-b793-5caaffc39884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:29.070650 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4fd101af-e9c3-457f-b793-5caaffc39884 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:29.071849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4fd101af-e9c3-457f-b793-5caaffc39884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:29.099688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:29.150168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 72/143] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:29.174778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8962e0-4071-4679-9d32-7745d74332f6 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:29.221476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:29.298304 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8962e0-4071-4679-9d32-7745d74332f6 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:29.332286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145ba2f-c7a4-44aa-a0fe-2c414107c90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:29.333085 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145ba2f-c7a4-44aa-a0fe-2c414107c90f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:29.333394 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145ba2f-c7a4-44aa-a0fe-2c414107c90f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:29.333668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145ba2f-c7a4-44aa-a0fe-2c414107c90f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:29.347378 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145ba2f-c7a4-44aa-a0fe-2c414107c90f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-c145ba2f-c7a4-44aa-a0fe-2c414107c90f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:29.355813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:29.356724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:29.357634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:29.357634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:29.357634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:29.357634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:29.357634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:29.357634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:29.357634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:29.357634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:29.357634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:29.357634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:29.357634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:29.357634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:29.357634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 72/144] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:29.366595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd101af-e9c3-457f-b793-5caaffc39884 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:29.394147 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59058555-e42a-4b05-be3b-6453600017fc None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:29.394460 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59058555-e42a-4b05-be3b-6453600017fc None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:29.394702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59058555-e42a-4b05-be3b-6453600017fc None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:29.396337 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 73/145] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:29.406273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94e6f3-1e7d-41c7-b335-85223986d1f2 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:29.406566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94e6f3-1e7d-41c7-b335-85223986d1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:29.406827 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94e6f3-1e7d-41c7-b335-85223986d1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:29.439342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-8a94e6f3-1e7d-41c7-b335-85223986d1f2 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:29.440644 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-8a94e6f3-1e7d-41c7-b335-85223986d1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:29.573327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd101af-e9c3-457f-b793-5caaffc39884 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:29.579019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 74/146] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:29.617412 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf841e-b5d8-40f1-96a3-4b6019d9eb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:29.618113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf841e-b5d8-40f1-96a3-4b6019d9eb01 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:29.618400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf841e-b5d8-40f1-96a3-4b6019d9eb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:29.618683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf841e-b5d8-40f1-96a3-4b6019d9eb01 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:29.620306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94e6f3-1e7d-41c7-b335-85223986d1f2 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:29.623401 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf841e-b5d8-40f1-96a3-4b6019d9eb01 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:29.623728 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf841e-b5d8-40f1-96a3-4b6019d9eb01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:29.629556 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf841e-b5d8-40f1-96a3-4b6019d9eb01 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:29.651030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 73/147] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:29.663924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf841e-b5d8-40f1-96a3-4b6019d9eb01 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:29.704004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:29.714280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:29.768360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94e6f3-1e7d-41c7-b335-85223986d1f2 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:29.796290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf841e-b5d8-40f1-96a3-4b6019d9eb01 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:29.807332 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7101e-d15f-4bcd-876b-715c39fc7a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:29.808159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7101e-d15f-4bcd-876b-715c39fc7a9f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:29.808438 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7101e-d15f-4bcd-876b-715c39fc7a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:29.808711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7101e-d15f-4bcd-876b-715c39fc7a9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:29.820239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7101e-d15f-4bcd-876b-715c39fc7a9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-78e7101e-d15f-4bcd-876b-715c39fc7a9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:29.835558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:29.836431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:29.837232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:29.837232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:29.837232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:29.837232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:29.837232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:29.837232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:29.837232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:29.837232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:29.837232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:29.837232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:29.837232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:29.837232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:29.837830 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 75/148] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:16:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:16:29.842821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a031b3-b91a-4627-b567-3c24c54c0ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:29.843861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a031b3-b91a-4627-b567-3c24c54c0ccd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:29.844146 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a031b3-b91a-4627-b567-3c24c54c0ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:29.844428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a031b3-b91a-4627-b567-3c24c54c0ccd None admin] PATH_INFO: `/v3/users/581c8d476f744f25ad957b35c68a5fe2` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:29.844592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7101e-d15f-4bcd-876b-715c39fc7a9f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:29.849611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a031b3-b91a-4627-b567-3c24c54c0ccd None admin] RBAC: Authorizing `identity:get_user(user_id=581c8d476f744f25ad957b35c68a5fe2)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:29.851489 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a031b3-b91a-4627-b567-3c24c54c0ccd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:29.853223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 74/149] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:29 2022] GET /identity/v3/users/581c8d476f744f25ad957b35c68a5fe2 => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:29.858658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a031b3-b91a-4627-b567-3c24c54c0ccd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:29.967409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:29.980031 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7101e-d15f-4bcd-876b-715c39fc7a9f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:29.986919 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:30.009068 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a031b3-b91a-4627-b567-3c24c54c0ccd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:30.026529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f2715-ffc1-484b-b3c5-5b3d30a0090a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:30.027215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f2715-ffc1-484b-b3c5-5b3d30a0090a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:30.027497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f2715-ffc1-484b-b3c5-5b3d30a0090a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:30.027772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f2715-ffc1-484b-b3c5-5b3d30a0090a None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:30.028784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f2715-ffc1-484b-b3c5-5b3d30a0090a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:30.029088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f2715-ffc1-484b-b3c5-5b3d30a0090a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:30.030569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f2715-ffc1-484b-b3c5-5b3d30a0090a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:30.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 76/150] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:16:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:30.046358 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f2715-ffc1-484b-b3c5-5b3d30a0090a None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:30.065562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1ff10-7b66-4782-81b8-b503716e37cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:30.066459 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1ff10-7b66-4782-81b8-b503716e37cc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:30.066850 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1ff10-7b66-4782-81b8-b503716e37cc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:30.067130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1ff10-7b66-4782-81b8-b503716e37cc None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:30.071223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1ff10-7b66-4782-81b8-b503716e37cc None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:30.072723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1ff10-7b66-4782-81b8-b503716e37cc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:30.074648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 75/151] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:29 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:30.082635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1ff10-7b66-4782-81b8-b503716e37cc None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:30.261244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f2715-ffc1-484b-b3c5-5b3d30a0090a None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:30.272188 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1ff10-7b66-4782-81b8-b503716e37cc None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:30.346445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1dc04-0eb0-4fea-a634-dd2afbe6c945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:30.347957 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1dc04-0eb0-4fea-a634-dd2afbe6c945 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:30.348433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1dc04-0eb0-4fea-a634-dd2afbe6c945 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:30.348899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1dc04-0eb0-4fea-a634-dd2afbe6c945 None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5/users/581c8d476f744f25ad957b35c68a5fe2/roles/64202be313d04a5aab01b866bc558152` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:30.351698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1dc04-0eb0-4fea-a634-dd2afbe6c945 None admin] RBAC: Authorizing `identity:create_grant(project_id=30d54bfc38f04330a2dc73fcb0219ec5, user_id=581c8d476f744f25ad957b35c68a5fe2, role_id=64202be313d04a5aab01b866bc558152)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:30.353891 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1dc04-0eb0-4fea-a634-dd2afbe6c945 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:30.357863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada25d39-1f38-477e-b8e0-b0d74f4c8dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:30.358855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada25d39-1f38-477e-b8e0-b0d74f4c8dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:30.359141 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada25d39-1f38-477e-b8e0-b0d74f4c8dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:30.359427 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada25d39-1f38-477e-b8e0-b0d74f4c8dd0 None admin] PATH_INFO: `/v3/users/285435c09061437080caaff56db1dced` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:30.360747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada25d39-1f38-477e-b8e0-b0d74f4c8dd0 None admin] RBAC: Authorizing `identity:get_user(user_id=285435c09061437080caaff56db1dced)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:30.362268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada25d39-1f38-477e-b8e0-b0d74f4c8dd0 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:30.363983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 77/152] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:30 2022] GET /identity/v3/users/285435c09061437080caaff56db1dced => generated 275 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:30.371055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada25d39-1f38-477e-b8e0-b0d74f4c8dd0 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:30.379924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 76/153] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:16:30 2022] PUT /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5/users/581c8d476f744f25ad957b35c68a5fe2/roles/64202be313d04a5aab01b866bc558152 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:16:30.425446 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:30.537370 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada25d39-1f38-477e-b8e0-b0d74f4c8dd0 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:30.591470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb75e2a-22c5-4a28-8a9f-e6b9427ff36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:30.592133 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb75e2a-22c5-4a28-8a9f-e6b9427ff36e None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:30.592415 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb75e2a-22c5-4a28-8a9f-e6b9427ff36e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:30.592691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb75e2a-22c5-4a28-8a9f-e6b9427ff36e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:30.681218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb75e2a-22c5-4a28-8a9f-e6b9427ff36e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:30.682734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb75e2a-22c5-4a28-8a9f-e6b9427ff36e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:30.713394 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:30.830610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 78/154] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 15:16:30.836615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1dc04-0eb0-4fea-a634-dd2afbe6c945 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:30.872052 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:30.934675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a005a-d925-40c7-8b6b-bb59cb73d003 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:30.935140 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a005a-d925-40c7-8b6b-bb59cb73d003 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:30.935249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a005a-d925-40c7-8b6b-bb59cb73d003 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:30.936893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 79/155] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:30.942307 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30073207-cdeb-4a9f-a01c-aa4bf93d6272 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:30.942548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30073207-cdeb-4a9f-a01c-aa4bf93d6272 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:30.942812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30073207-cdeb-4a9f-a01c-aa4bf93d6272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:30.982287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-30073207-cdeb-4a9f-a01c-aa4bf93d6272 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:30.983853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-30073207-cdeb-4a9f-a01c-aa4bf93d6272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:30.994013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1dc04-0eb0-4fea-a634-dd2afbe6c945 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:31.050605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54299ae-df29-4244-9120-6e09c6663ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:31.051831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54299ae-df29-4244-9120-6e09c6663ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:31.052430 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54299ae-df29-4244-9120-6e09c6663ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:31.052896 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54299ae-df29-4244-9120-6e09c6663ba6 None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:31.058219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54299ae-df29-4244-9120-6e09c6663ba6 None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:31.060082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54299ae-df29-4244-9120-6e09c6663ba6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:31.062583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 77/156] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:30 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:31.064273 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86574e-d43d-466e-b8ea-3f520442e6c9 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:31.064880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86574e-d43d-466e-b8ea-3f520442e6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:31.065401 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86574e-d43d-466e-b8ea-3f520442e6c9 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:31.067209 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 78/157] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:31.076856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4643d-a200-4b47-878d-4ebf5a24058b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:31.077623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4643d-a200-4b47-878d-4ebf5a24058b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:31.078305 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4643d-a200-4b47-878d-4ebf5a24058b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:31.115714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-05b4643d-a200-4b47-878d-4ebf5a24058b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:31.117475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-05b4643d-a200-4b47-878d-4ebf5a24058b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:31.157247 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 80/158] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:31.158898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30073207-cdeb-4a9f-a01c-aa4bf93d6272 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:31.262703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:31.301268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 79/159] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:31.302138 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4643d-a200-4b47-878d-4ebf5a24058b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:31.323722 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30073207-cdeb-4a9f-a01c-aa4bf93d6272 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:31.330220 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:31.364687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86417c-7404-49b1-9371-735473035fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:31.365417 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86417c-7404-49b1-9371-735473035fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:31.367974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86417c-7404-49b1-9371-735473035fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:31.368230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86417c-7404-49b1-9371-735473035fd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:31.369920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86417c-7404-49b1-9371-735473035fd9 None admin] RBAC: Adding query filter params (role.id=44bd561cc5594ffd9bcc27ef6dad0fc5, scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5, user.id=285435c09061437080caaff56db1dced) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:31.370243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86417c-7404-49b1-9371-735473035fd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:31.372485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86417c-7404-49b1-9371-735473035fd9 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:31.388409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 81/160] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:31 2022] GET /identity/v3/role_assignments?user.id=285435c09061437080caaff56db1dced&scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5&role.id=44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:31.389291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86417c-7404-49b1-9371-735473035fd9 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:31.525459 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4643d-a200-4b47-878d-4ebf5a24058b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:31.565982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86417c-7404-49b1-9371-735473035fd9 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:31.600101 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:31.600442 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39263f5f-4b53-4bdc-9257-5e39c93f117b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:31.601946 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39263f5f-4b53-4bdc-9257-5e39c93f117b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:31.602298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39263f5f-4b53-4bdc-9257-5e39c93f117b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:31.602557 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39263f5f-4b53-4bdc-9257-5e39c93f117b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:31.624319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39263f5f-4b53-4bdc-9257-5e39c93f117b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-39263f5f-4b53-4bdc-9257-5e39c93f117b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:31.635761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:31.636723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:31.637520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:31.637520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:31.637520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:31.637520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:31.637520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:31.637520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:31.637520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:31.637520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:31.637520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:31.637520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:31.637520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:31.637520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:31.644053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61bac3-198e-4cc3-bc4d-176f415f6597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:31.644945 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61bac3-198e-4cc3-bc4d-176f415f6597 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:31.645288 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61bac3-198e-4cc3-bc4d-176f415f6597 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:31.645574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61bac3-198e-4cc3-bc4d-176f415f6597 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:31.646650 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 80/161] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:31.653528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39263f5f-4b53-4bdc-9257-5e39c93f117b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:31.676062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61bac3-198e-4cc3-bc4d-176f415f6597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-cc61bac3-198e-4cc3-bc4d-176f415f6597 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:31.690281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:31.692696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:31.695322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:31.695322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:31.695322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:31.695322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:31.695322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:31.695322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:31.695322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:31.695322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:31.695322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:31.695322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:31.695322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:31.695322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:31.695322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 82/162] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:16:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:31.702817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61bac3-198e-4cc3-bc4d-176f415f6597 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:31.833015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:31.892180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39263f5f-4b53-4bdc-9257-5e39c93f117b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:31.916318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61bac3-198e-4cc3-bc4d-176f415f6597 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:31.970157 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38959b04-2c7d-4adb-aaf0-b666b8b9fe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:31.970847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38959b04-2c7d-4adb-aaf0-b666b8b9fe38 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:31.971185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38959b04-2c7d-4adb-aaf0-b666b8b9fe38 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:31.971471 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38959b04-2c7d-4adb-aaf0-b666b8b9fe38 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:31.972242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38959b04-2c7d-4adb-aaf0-b666b8b9fe38 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:31.972554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38959b04-2c7d-4adb-aaf0-b666b8b9fe38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:31.977480 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38959b04-2c7d-4adb-aaf0-b666b8b9fe38 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:31.986703 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 81/163] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:31.987930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb4470-38f9-4f7a-b22b-e6266497e882 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:31.989030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb4470-38f9-4f7a-b22b-e6266497e882 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:31.989336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb4470-38f9-4f7a-b22b-e6266497e882 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:31.990737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 82/164] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:31.999153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb4470-38f9-4f7a-b22b-e6266497e882 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:32.000699 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469b791-5201-47ec-a2af-93d1053eacb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:32.001426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469b791-5201-47ec-a2af-93d1053eacb9 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:32.001712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469b791-5201-47ec-a2af-93d1053eacb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:32.002697 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469b791-5201-47ec-a2af-93d1053eacb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:32.003487 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469b791-5201-47ec-a2af-93d1053eacb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:32.005701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469b791-5201-47ec-a2af-93d1053eacb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:32.007324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469b791-5201-47ec-a2af-93d1053eacb9 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:32.021325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 83/165] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:32.024819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340752c3-d390-4cae-a09b-433bbd273ec1 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:32.025120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340752c3-d390-4cae-a09b-433bbd273ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:32.025420 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340752c3-d390-4cae-a09b-433bbd273ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:32.059413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-340752c3-d390-4cae-a09b-433bbd273ec1 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:32.060619 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-340752c3-d390-4cae-a09b-433bbd273ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:32.099154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:32.167690 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 84/166] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:32.168674 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340752c3-d390-4cae-a09b-433bbd273ec1 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:32.271801 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb4470-38f9-4f7a-b22b-e6266497e882 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:32.341816 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1f6bd-2eba-49e5-bbe5-4e39c0bbbc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:32.342700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1f6bd-2eba-49e5-bbe5-4e39c0bbbc7d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:32.342977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1f6bd-2eba-49e5-bbe5-4e39c0bbbc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:32.343246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1f6bd-2eba-49e5-bbe5-4e39c0bbbc7d None admin] PATH_INFO: `/v3/users/285435c09061437080caaff56db1dced` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:32.344561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1f6bd-2eba-49e5-bbe5-4e39c0bbbc7d None admin] RBAC: Authorizing `identity:get_user(user_id=285435c09061437080caaff56db1dced)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:32.346051 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1f6bd-2eba-49e5-bbe5-4e39c0bbbc7d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:32.347689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 83/167] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:31 2022] GET /identity/v3/users/285435c09061437080caaff56db1dced => generated 275 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:32.348525 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1f6bd-2eba-49e5-bbe5-4e39c0bbbc7d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:32.356684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340752c3-d390-4cae-a09b-433bbd273ec1 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:32.377876 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:32.459350 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca6046-30e6-40f2-bf51-e161b39063ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:32.460747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca6046-30e6-40f2-bf51-e161b39063ad None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:32.461011 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca6046-30e6-40f2-bf51-e161b39063ad None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:32.461295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca6046-30e6-40f2-bf51-e161b39063ad None admin] PATH_INFO: `/v3/users/669959f6d82b46b1b551b6c19bf21c50` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:32.462843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca6046-30e6-40f2-bf51-e161b39063ad None admin] RBAC: Authorizing `identity:get_user(user_id=669959f6d82b46b1b551b6c19bf21c50)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:32.464331 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca6046-30e6-40f2-bf51-e161b39063ad None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:32.466176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 85/168] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:32 2022] GET /identity/v3/users/669959f6d82b46b1b551b6c19bf21c50 => generated 247 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:32.467938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca6046-30e6-40f2-bf51-e161b39063ad None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:32.535281 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1f6bd-2eba-49e5-bbe5-4e39c0bbbc7d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:32.555595 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:32.608853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d698ee-fdd3-43b0-af60-6976fb0a93b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:32.609701 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d698ee-fdd3-43b0-af60-6976fb0a93b8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:32.609990 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d698ee-fdd3-43b0-af60-6976fb0a93b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:32.610273 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d698ee-fdd3-43b0-af60-6976fb0a93b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:32.637981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d698ee-fdd3-43b0-af60-6976fb0a93b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-93d698ee-fdd3-43b0-af60-6976fb0a93b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:32.646024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:32.647014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:32.647884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:32.647884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:32.647884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:32.647884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:32.647884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:32.647884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:32.647884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:32.647884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:32.647884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:32.647884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:32.647884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:32.647884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:32.647884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 84/169] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:32.648559 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d698ee-fdd3-43b0-af60-6976fb0a93b8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:32.689455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca6046-30e6-40f2-bf51-e161b39063ad None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:32.758151 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3557c-b86c-4e3e-8b55-a2cfe884b75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:32.758828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3557c-b86c-4e3e-8b55-a2cfe884b75d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:32.759114 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3557c-b86c-4e3e-8b55-a2cfe884b75d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:32.759392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3557c-b86c-4e3e-8b55-a2cfe884b75d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:32.777475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:32.847232 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d698ee-fdd3-43b0-af60-6976fb0a93b8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:32.896296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3557c-b86c-4e3e-8b55-a2cfe884b75d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:32.898351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3557c-b86c-4e3e-8b55-a2cfe884b75d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:32.929893 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094d345-a676-4043-8409-192a050f0f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:32.930568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094d345-a676-4043-8409-192a050f0f26 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:32.930851 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094d345-a676-4043-8409-192a050f0f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:32.931127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094d345-a676-4043-8409-192a050f0f26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:32.939218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:33.011166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 86/170] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:33.012193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3557c-b86c-4e3e-8b55-a2cfe884b75d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:33.119604 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3557c-b86c-4e3e-8b55-a2cfe884b75d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:33.128310 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094d345-a676-4043-8409-192a050f0f26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:33.130711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094d345-a676-4043-8409-192a050f0f26 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:33.155084 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540eedc8-7a82-4cc0-9207-01d35bf4e4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:33.155752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eedc8-7a82-4cc0-9207-01d35bf4e4fc None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:33.156042 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eedc8-7a82-4cc0-9207-01d35bf4e4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:33.156317 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eedc8-7a82-4cc0-9207-01d35bf4e4fc None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:33.157074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540eedc8-7a82-4cc0-9207-01d35bf4e4fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:33.157424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540eedc8-7a82-4cc0-9207-01d35bf4e4fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:33.158798 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540eedc8-7a82-4cc0-9207-01d35bf4e4fc None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:33.160354 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:33.170574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 87/171] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:33.171936 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86e4f2-59e3-4935-993b-a94db3d5422f None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:33.172290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86e4f2-59e3-4935-993b-a94db3d5422f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:33.172634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86e4f2-59e3-4935-993b-a94db3d5422f None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:33.174112 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 88/172] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:33.174744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86e4f2-59e3-4935-993b-a94db3d5422f None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:33.218143 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:33.274508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 85/173] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:33.275800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f244214-8765-4aa4-ba0f-27d275203a5f None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:33.276092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f244214-8765-4aa4-ba0f-27d275203a5f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:33.276373 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f244214-8765-4aa4-ba0f-27d275203a5f None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:33.278265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 86/174] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:33.282528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b0f02-3d72-49fa-ae2f-c4f4a6150978 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:33.283098 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b0f02-3d72-49fa-ae2f-c4f4a6150978 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:33.283376 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b0f02-3d72-49fa-ae2f-c4f4a6150978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:33.298302 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86e4f2-59e3-4935-993b-a94db3d5422f None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:33.304457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-834b0f02-3d72-49fa-ae2f-c4f4a6150978 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:33.305662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-834b0f02-3d72-49fa-ae2f-c4f4a6150978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:33.337542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87d8bd-707f-4fd4-a236-113ebbc752fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:33.338404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87d8bd-707f-4fd4-a236-113ebbc752fb None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:33.338685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87d8bd-707f-4fd4-a236-113ebbc752fb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:33.338959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87d8bd-707f-4fd4-a236-113ebbc752fb None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:33.340249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87d8bd-707f-4fd4-a236-113ebbc752fb None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:33.342006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87d8bd-707f-4fd4-a236-113ebbc752fb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:33.348537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 89/175] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:33 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:33.348537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87d8bd-707f-4fd4-a236-113ebbc752fb None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:33.374921 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 87/176] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:33.375775 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b0f02-3d72-49fa-ae2f-c4f4a6150978 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:33.424047 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:33.465515 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87d8bd-707f-4fd4-a236-113ebbc752fb None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:33.484114 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:33.493453 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b0f02-3d72-49fa-ae2f-c4f4a6150978 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:33.500023 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b57d3-e1e6-497a-a0ce-ecf2c337af9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:33.500895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b57d3-e1e6-497a-a0ce-ecf2c337af9a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:33.501205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b57d3-e1e6-497a-a0ce-ecf2c337af9a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:33.501549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b57d3-e1e6-497a-a0ce-ecf2c337af9a None admin] PATH_INFO: `/v3/users/285435c09061437080caaff56db1dced` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:33.502761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b57d3-e1e6-497a-a0ce-ecf2c337af9a None admin] RBAC: Authorizing `identity:get_user(user_id=285435c09061437080caaff56db1dced)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:33.504218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b57d3-e1e6-497a-a0ce-ecf2c337af9a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:33.506058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 90/177] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:33 2022] GET /identity/v3/users/285435c09061437080caaff56db1dced => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:33.507589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797af97-2e40-4c2c-9e9d-9d4aed70ec49 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:33.507865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797af97-2e40-4c2c-9e9d-9d4aed70ec49 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:33.508137 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797af97-2e40-4c2c-9e9d-9d4aed70ec49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:33.531015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-f797af97-2e40-4c2c-9e9d-9d4aed70ec49 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:33.532113 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f797af97-2e40-4c2c-9e9d-9d4aed70ec49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:33.536974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf34bd-24c9-40e2-8b05-2326ec55010b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:33.537873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf34bd-24c9-40e2-8b05-2326ec55010b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:33.538311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf34bd-24c9-40e2-8b05-2326ec55010b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:33.539083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf34bd-24c9-40e2-8b05-2326ec55010b None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:33.540297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf34bd-24c9-40e2-8b05-2326ec55010b None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:33.541783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf34bd-24c9-40e2-8b05-2326ec55010b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:33.543668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 88/178] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:33 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:33.549609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82606afa-4b59-4a54-a8b5-2f46efab9ca0 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:33.549899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82606afa-4b59-4a54-a8b5-2f46efab9ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:33.550178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82606afa-4b59-4a54-a8b5-2f46efab9ca0 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:33.551576 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 89/179] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:33.552145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82606afa-4b59-4a54-a8b5-2f46efab9ca0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:33.607809 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 91/180] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:33.608478 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797af97-2e40-4c2c-9e9d-9d4aed70ec49 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:33.680152 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:33.688520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82606afa-4b59-4a54-a8b5-2f46efab9ca0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:33.723686 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797af97-2e40-4c2c-9e9d-9d4aed70ec49 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:33.728027 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38ae21-6bef-405a-8b11-6e592a537880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:33.728696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38ae21-6bef-405a-8b11-6e592a537880 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:33.729008 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38ae21-6bef-405a-8b11-6e592a537880 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:33.729292 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38ae21-6bef-405a-8b11-6e592a537880 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:33.730140 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38ae21-6bef-405a-8b11-6e592a537880 None admin] RBAC: Adding query filter params (role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f, scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5, user.id=285435c09061437080caaff56db1dced) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:33.730461 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38ae21-6bef-405a-8b11-6e592a537880 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:33.731919 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38ae21-6bef-405a-8b11-6e592a537880 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:33.738488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 90/181] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:33 2022] GET /identity/v3/role_assignments?user.id=285435c09061437080caaff56db1dced&scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5&role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:33.739031 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38ae21-6bef-405a-8b11-6e592a537880 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:33.773802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:33.781981 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79e6aa-237b-48ee-9fe2-a26ed8b0a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:33.784662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79e6aa-237b-48ee-9fe2-a26ed8b0a77b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:33.785195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79e6aa-237b-48ee-9fe2-a26ed8b0a77b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:33.785656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79e6aa-237b-48ee-9fe2-a26ed8b0a77b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:33.802779 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79e6aa-237b-48ee-9fe2-a26ed8b0a77b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-6b79e6aa-237b-48ee-9fe2-a26ed8b0a77b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:33.814636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:33.815600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:33.816375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:33.816375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:33.816375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:33.816375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:33.816375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:33.816375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:33.816375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:33.816375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:33.816375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:33.816375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:33.816375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:33.816375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:33.818810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 92/182] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:16:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:33.820500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79e6aa-237b-48ee-9fe2-a26ed8b0a77b None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:33.872885 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38ae21-6bef-405a-8b11-6e592a537880 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:33.918020 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1784a-9765-461f-8d70-a4650d43fd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:33.919032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1784a-9765-461f-8d70-a4650d43fd81 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:33.919032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1784a-9765-461f-8d70-a4650d43fd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:33.919238 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1784a-9765-461f-8d70-a4650d43fd81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:33.950904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79e6aa-237b-48ee-9fe2-a26ed8b0a77b None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:33.978652 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:33.990335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1145a-58e4-4d80-bb92-5149f3f9dcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:33.991055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1145a-58e4-4d80-bb92-5149f3f9dcde None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:33.991953 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1145a-58e4-4d80-bb92-5149f3f9dcde None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:33.992303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1145a-58e4-4d80-bb92-5149f3f9dcde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:33.993278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1145a-58e4-4d80-bb92-5149f3f9dcde None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5, user.id=669959f6d82b46b1b551b6c19bf21c50) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:33.993634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1145a-58e4-4d80-bb92-5149f3f9dcde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:33.995089 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1145a-58e4-4d80-bb92-5149f3f9dcde None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:33.998952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1784a-9765-461f-8d70-a4650d43fd81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:34.000409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1784a-9765-461f-8d70-a4650d43fd81 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:34.001148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 93/183] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:33 2022] GET /identity/v3/role_assignments?user.id=669959f6d82b46b1b551b6c19bf21c50&scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:34.002341 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef235e-fd4a-484f-9587-142dad4096fd None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:34.002630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef235e-fd4a-484f-9587-142dad4096fd None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:34.002911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef235e-fd4a-484f-9587-142dad4096fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:34.027313 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ebef235e-fd4a-484f-9587-142dad4096fd None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:34.028808 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ebef235e-fd4a-484f-9587-142dad4096fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:34.029657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:34.107981 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 94/184] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:34.108597 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef235e-fd4a-484f-9587-142dad4096fd None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:34.129753 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 91/185] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:34.130957 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1784a-9765-461f-8d70-a4650d43fd81 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:34.231850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef235e-fd4a-484f-9587-142dad4096fd None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:34.249704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:34.264265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1784a-9765-461f-8d70-a4650d43fd81 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:34.268237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05dafa4-e15c-4ee1-8baa-eba6ecc186de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:34.269041 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05dafa4-e15c-4ee1-8baa-eba6ecc186de None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:34.269345 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05dafa4-e15c-4ee1-8baa-eba6ecc186de None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:34.269619 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05dafa4-e15c-4ee1-8baa-eba6ecc186de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:34.280012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05dafa4-e15c-4ee1-8baa-eba6ecc186de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-d05dafa4-e15c-4ee1-8baa-eba6ecc186de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:34.288721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:34.289728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:34.290502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:34.290502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:34.290502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:34.290502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:34.290502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:34.290502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:34.290502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:34.290502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:34.290502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:34.290502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:34.290502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:34.290502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:34.290502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 95/186] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:16:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:34.291655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63664918-b850-40a0-9bb6-4e4b9fd8baf4 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:34.291831 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63664918-b850-40a0-9bb6-4e4b9fd8baf4 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:34.292090 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63664918-b850-40a0-9bb6-4e4b9fd8baf4 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:34.297172 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 96/187] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:34.297172 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63664918-b850-40a0-9bb6-4e4b9fd8baf4 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:34.301721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:34.308861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234e371-36e3-4237-8b3d-69c6f0b5caa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:34.309568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234e371-36e3-4237-8b3d-69c6f0b5caa1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:34.309853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234e371-36e3-4237-8b3d-69c6f0b5caa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:34.310129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234e371-36e3-4237-8b3d-69c6f0b5caa1 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:34.310893 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234e371-36e3-4237-8b3d-69c6f0b5caa1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:34.311205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234e371-36e3-4237-8b3d-69c6f0b5caa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:34.318107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234e371-36e3-4237-8b3d-69c6f0b5caa1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:34.327128 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 92/188] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:34.327917 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234e371-36e3-4237-8b3d-69c6f0b5caa1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:34.411984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63664918-b850-40a0-9bb6-4e4b9fd8baf4 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:34.430813 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234e371-36e3-4237-8b3d-69c6f0b5caa1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:34.445832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc485d6-8a47-427a-826b-1e783fa6d1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:34.446620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc485d6-8a47-427a-826b-1e783fa6d1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:34.446895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc485d6-8a47-427a-826b-1e783fa6d1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:34.447174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc485d6-8a47-427a-826b-1e783fa6d1d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:34.448329 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc485d6-8a47-427a-826b-1e783fa6d1d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:34.449826 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc485d6-8a47-427a-826b-1e783fa6d1d2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:34.451439 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 97/189] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:34.452199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc485d6-8a47-427a-826b-1e783fa6d1d2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:34.467284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dbcdc-800c-4ee4-8148-daa162a81680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:34.468148 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dbcdc-800c-4ee4-8148-daa162a81680 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:34.468428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dbcdc-800c-4ee4-8148-daa162a81680 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:34.468705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dbcdc-800c-4ee4-8148-daa162a81680 None admin] PATH_INFO: `/v3/projects/fc3a144aaec74055a742212b2a2c746f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:34.469902 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981dbcdc-800c-4ee4-8148-daa162a81680 None admin] RBAC: Authorizing `identity:get_project(project_id=fc3a144aaec74055a742212b2a2c746f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:34.471352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981dbcdc-800c-4ee4-8148-daa162a81680 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:34.473047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 93/190] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:34 2022] GET /identity/v3/projects/fc3a144aaec74055a742212b2a2c746f => generated 312 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:34.474191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f359fc7-f5cf-481b-82b8-4f57258a2b49 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:34.474473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f359fc7-f5cf-481b-82b8-4f57258a2b49 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:34.474745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f359fc7-f5cf-481b-82b8-4f57258a2b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:34.491026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0f359fc7-f5cf-481b-82b8-4f57258a2b49 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:34.492924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0f359fc7-f5cf-481b-82b8-4f57258a2b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:34.502183 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:34.542191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:34.555983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc485d6-8a47-427a-826b-1e783fa6d1d2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:34.559117 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 94/191] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:34.559948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f359fc7-f5cf-481b-82b8-4f57258a2b49 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:34.589856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8379a-22bf-4acb-94e3-8b2c00353e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:34.590523 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8379a-22bf-4acb-94e3-8b2c00353e80 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:34.590796 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8379a-22bf-4acb-94e3-8b2c00353e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:34.591067 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8379a-22bf-4acb-94e3-8b2c00353e80 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:34.591812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8379a-22bf-4acb-94e3-8b2c00353e80 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:34.592117 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8379a-22bf-4acb-94e3-8b2c00353e80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:34.593524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8379a-22bf-4acb-94e3-8b2c00353e80 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:34.602014 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 98/192] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:16:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:34.602777 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8379a-22bf-4acb-94e3-8b2c00353e80 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:34.661672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f359fc7-f5cf-481b-82b8-4f57258a2b49 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:34.695397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf19dbd-ddfc-4c85-b64d-03569755bdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:34.696188 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf19dbd-ddfc-4c85-b64d-03569755bdac None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:34.696470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf19dbd-ddfc-4c85-b64d-03569755bdac None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:34.696751 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf19dbd-ddfc-4c85-b64d-03569755bdac None admin] PATH_INFO: `/v3/users/f23c7568c8834d65988a7cbd39a5f0d2` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:34.700542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8379a-22bf-4acb-94e3-8b2c00353e80 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:34.707368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:34.710273 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf19dbd-ddfc-4c85-b64d-03569755bdac None admin] RBAC: Authorizing `identity:get_user(user_id=f23c7568c8834d65988a7cbd39a5f0d2)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:34.711821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf19dbd-ddfc-4c85-b64d-03569755bdac None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:34.713559 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 95/193] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:34 2022] GET /identity/v3/users/f23c7568c8834d65988a7cbd39a5f0d2 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:34.714331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf19dbd-ddfc-4c85-b64d-03569755bdac None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:34.738584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16794759-5fde-4b8d-9ed3-d590dca3bbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:34.739251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16794759-5fde-4b8d-9ed3-d590dca3bbbb None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:34.739536 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16794759-5fde-4b8d-9ed3-d590dca3bbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:34.739813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16794759-5fde-4b8d-9ed3-d590dca3bbbb None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:34.740550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16794759-5fde-4b8d-9ed3-d590dca3bbbb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:34.742530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16794759-5fde-4b8d-9ed3-d590dca3bbbb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:34.760879 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:34.761314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 99/194] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:16:34 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:16:34.762121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16794759-5fde-4b8d-9ed3-d590dca3bbbb None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:34.833670 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf19dbd-ddfc-4c85-b64d-03569755bdac None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:34.870938 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff7cec-5f57-4b76-8e4c-eb80d9b9408b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:34.871620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff7cec-5f57-4b76-8e4c-eb80d9b9408b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:34.871906 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff7cec-5f57-4b76-8e4c-eb80d9b9408b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:34.872186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff7cec-5f57-4b76-8e4c-eb80d9b9408b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:34.873056 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff7cec-5f57-4b76-8e4c-eb80d9b9408b None admin] RBAC: Adding query filter params (role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f, scope.project.id=fc3a144aaec74055a742212b2a2c746f, user.id=285435c09061437080caaff56db1dced) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:34.873409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff7cec-5f57-4b76-8e4c-eb80d9b9408b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:34.874915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff7cec-5f57-4b76-8e4c-eb80d9b9408b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:34.879513 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16794759-5fde-4b8d-9ed3-d590dca3bbbb None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:34.881372 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 96/195] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:34 2022] GET /identity/v3/role_assignments?user.id=285435c09061437080caaff56db1dced&scope.project.id=fc3a144aaec74055a742212b2a2c746f&role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:34.883051 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff7cec-5f57-4b76-8e4c-eb80d9b9408b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:34.939589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f44d9-2e21-4332-89d6-8967fa780024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:34.941101 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f44d9-2e21-4332-89d6-8967fa780024 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:34.941453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f44d9-2e21-4332-89d6-8967fa780024 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:34.941724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f44d9-2e21-4332-89d6-8967fa780024 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:34.959614 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f44d9-2e21-4332-89d6-8967fa780024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-143f44d9-2e21-4332-89d6-8967fa780024 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:34.969052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:34.970035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:34.970836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:34.970836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:34.970836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:34.970836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:34.970836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:34.970836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:34.970836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:34.970836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:34.970836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:34.970836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:34.970836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:34.970836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:34.970836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 100/196] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:34.971483 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f44d9-2e21-4332-89d6-8967fa780024 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:35.001963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:35.028034 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff7cec-5f57-4b76-8e4c-eb80d9b9408b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:35.039761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:35.075588 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52847304-a6f9-4521-b5e9-c7d74927cd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:35.076487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52847304-a6f9-4521-b5e9-c7d74927cd12 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:35.076771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52847304-a6f9-4521-b5e9-c7d74927cd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:35.077050 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52847304-a6f9-4521-b5e9-c7d74927cd12 None admin] PATH_INFO: `/v3/users/f23c7568c8834d65988a7cbd39a5f0d2` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:35.078576 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52847304-a6f9-4521-b5e9-c7d74927cd12 None admin] RBAC: Authorizing `identity:get_user(user_id=f23c7568c8834d65988a7cbd39a5f0d2)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:35.080040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52847304-a6f9-4521-b5e9-c7d74927cd12 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:35.082080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 97/197] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:34 2022] GET /identity/v3/users/f23c7568c8834d65988a7cbd39a5f0d2 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:35.082725 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52847304-a6f9-4521-b5e9-c7d74927cd12 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:35.126791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f44d9-2e21-4332-89d6-8967fa780024 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:35.170758 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42890b30-ce1a-41f6-a5fc-85d29dd3ac76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:35.171665 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42890b30-ce1a-41f6-a5fc-85d29dd3ac76 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:35.171951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42890b30-ce1a-41f6-a5fc-85d29dd3ac76 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:35.172228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42890b30-ce1a-41f6-a5fc-85d29dd3ac76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:35.230738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52847304-a6f9-4521-b5e9-c7d74927cd12 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:35.271773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42890b30-ce1a-41f6-a5fc-85d29dd3ac76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:35.274372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42890b30-ce1a-41f6-a5fc-85d29dd3ac76 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:35.281831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b856622-05ba-435d-86e7-701629823b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:35.282845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b856622-05ba-435d-86e7-701629823b28 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:35.283869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b856622-05ba-435d-86e7-701629823b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:35.284345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b856622-05ba-435d-86e7-701629823b28 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:35.285782 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b856622-05ba-435d-86e7-701629823b28 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:35.286312 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b856622-05ba-435d-86e7-701629823b28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:35.288678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b856622-05ba-435d-86e7-701629823b28 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:35.299817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:35.301693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 98/198] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:35.303175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b856622-05ba-435d-86e7-701629823b28 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:35.348941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:35.413877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 101/199] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:35.414853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b856622-05ba-435d-86e7-701629823b28 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:35.415128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42890b30-ce1a-41f6-a5fc-85d29dd3ac76 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:35.457028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb357d7f-5392-4128-aaf2-ef1739ef55ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:35.457943 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb357d7f-5392-4128-aaf2-ef1739ef55ed None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:35.458366 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb357d7f-5392-4128-aaf2-ef1739ef55ed None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:35.458800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb357d7f-5392-4128-aaf2-ef1739ef55ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:35.536246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb357d7f-5392-4128-aaf2-ef1739ef55ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:35.537533 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb357d7f-5392-4128-aaf2-ef1739ef55ed None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:35.543592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42890b30-ce1a-41f6-a5fc-85d29dd3ac76 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:35.576490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:35.580332 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:35.593351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d300a8-bc6d-4357-945a-a6325dc004f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:35.595033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d300a8-bc6d-4357-945a-a6325dc004f1 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:35.595479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d300a8-bc6d-4357-945a-a6325dc004f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:35.595894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d300a8-bc6d-4357-945a-a6325dc004f1 None admin] PATH_INFO: `/v3/users/581c8d476f744f25ad957b35c68a5fe2` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:35.597280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d300a8-bc6d-4357-945a-a6325dc004f1 None admin] RBAC: Authorizing `identity:get_user(user_id=581c8d476f744f25ad957b35c68a5fe2)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:35.599727 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d300a8-bc6d-4357-945a-a6325dc004f1 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:35.602868 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 102/200] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:35 2022] GET /identity/v3/users/581c8d476f744f25ad957b35c68a5fe2 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:35.604126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d300a8-bc6d-4357-945a-a6325dc004f1 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:35.656342 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 99/201] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:35.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb357d7f-5392-4128-aaf2-ef1739ef55ed None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:35.724485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d300a8-bc6d-4357-945a-a6325dc004f1 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:35.761804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c331f-67fa-4936-b220-b3e78d94ca28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:35.762765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c331f-67fa-4936-b220-b3e78d94ca28 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:35.763115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c331f-67fa-4936-b220-b3e78d94ca28 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:35.763477 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c331f-67fa-4936-b220-b3e78d94ca28 None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:35.764742 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c331f-67fa-4936-b220-b3e78d94ca28 None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:35.766290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c331f-67fa-4936-b220-b3e78d94ca28 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:35.768236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 103/202] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:35 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:35.768955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c331f-67fa-4936-b220-b3e78d94ca28 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:35.792653 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb357d7f-5392-4128-aaf2-ef1739ef55ed None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:35.834511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:35.843382 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da449cd-43d2-421c-bf1c-b457a4ba4f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:35.844546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da449cd-43d2-421c-bf1c-b457a4ba4f60 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:35.845568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da449cd-43d2-421c-bf1c-b457a4ba4f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:35.846564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da449cd-43d2-421c-bf1c-b457a4ba4f60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:35.858529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:35.883273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c331f-67fa-4936-b220-b3e78d94ca28 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:35.931562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0cdc5-c88e-4d79-8738-c7fd6b9dc531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:35.932445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0cdc5-c88e-4d79-8738-c7fd6b9dc531 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:35.932727 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0cdc5-c88e-4d79-8738-c7fd6b9dc531 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:35.933018 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0cdc5-c88e-4d79-8738-c7fd6b9dc531 None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:35.934424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0cdc5-c88e-4d79-8738-c7fd6b9dc531 None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:35.935872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0cdc5-c88e-4d79-8738-c7fd6b9dc531 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:35.937904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 104/203] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:35 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:35.938999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0cdc5-c88e-4d79-8738-c7fd6b9dc531 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:35.954973 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da449cd-43d2-421c-bf1c-b457a4ba4f60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:35.957409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da449cd-43d2-421c-bf1c-b457a4ba4f60 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:36.060692 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 100/204] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:36.065208 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da449cd-43d2-421c-bf1c-b457a4ba4f60 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:36.067402 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0cdc5-c88e-4d79-8738-c7fd6b9dc531 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:36.094227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:36.105442 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503e2a5-ff1f-48dc-b0b8-2313a6a71bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:36.106373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503e2a5-ff1f-48dc-b0b8-2313a6a71bda None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:36.106886 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503e2a5-ff1f-48dc-b0b8-2313a6a71bda None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:36.107333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503e2a5-ff1f-48dc-b0b8-2313a6a71bda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:36.108375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503e2a5-ff1f-48dc-b0b8-2313a6a71bda None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=2bda471206fe4fbb88370df9ab3bdfac, user.id=f23c7568c8834d65988a7cbd39a5f0d2) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:36.108912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503e2a5-ff1f-48dc-b0b8-2313a6a71bda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:36.110607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503e2a5-ff1f-48dc-b0b8-2313a6a71bda None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:36.113518 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:36.118434 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 105/205] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:35 2022] GET /identity/v3/role_assignments?user.id=f23c7568c8834d65988a7cbd39a5f0d2&scope.project.id=2bda471206fe4fbb88370df9ab3bdfac&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:36.126489 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503e2a5-ff1f-48dc-b0b8-2313a6a71bda None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:36.171734 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da449cd-43d2-421c-bf1c-b457a4ba4f60 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:36.217443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75d801-bc7c-4a55-a099-cf09bc112436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:36.218141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75d801-bc7c-4a55-a099-cf09bc112436 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:36.218428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75d801-bc7c-4a55-a099-cf09bc112436 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:36.218712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75d801-bc7c-4a55-a099-cf09bc112436 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:36.220230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75d801-bc7c-4a55-a099-cf09bc112436 None admin] RBAC: Adding query filter params (role.id=44bd561cc5594ffd9bcc27ef6dad0fc5, scope.project.id=2bda471206fe4fbb88370df9ab3bdfac, user.id=f23c7568c8834d65988a7cbd39a5f0d2) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:36.220555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75d801-bc7c-4a55-a099-cf09bc112436 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:36.222321 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75d801-bc7c-4a55-a099-cf09bc112436 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:36.230522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 101/206] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:36 2022] GET /identity/v3/role_assignments?user.id=f23c7568c8834d65988a7cbd39a5f0d2&scope.project.id=2bda471206fe4fbb88370df9ab3bdfac&role.id=44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:36.357210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503e2a5-ff1f-48dc-b0b8-2313a6a71bda None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:36.409812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d64a7-21ff-4b86-99f0-f22fe68ea997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:36.410686 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d64a7-21ff-4b86-99f0-f22fe68ea997 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:36.410967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d64a7-21ff-4b86-99f0-f22fe68ea997 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:36.411243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d64a7-21ff-4b86-99f0-f22fe68ea997 None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:36.412496 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d64a7-21ff-4b86-99f0-f22fe68ea997 None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:36.414600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d64a7-21ff-4b86-99f0-f22fe68ea997 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:36.416382 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 106/207] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:36 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:36.426293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75d801-bc7c-4a55-a099-cf09bc112436 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:36.533898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75d801-bc7c-4a55-a099-cf09bc112436 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:36.535443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:36.583087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c478b62-2835-4bef-bda2-660e1c0d9e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:36.584103 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c478b62-2835-4bef-bda2-660e1c0d9e23 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:36.584748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c478b62-2835-4bef-bda2-660e1c0d9e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:36.585874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c478b62-2835-4bef-bda2-660e1c0d9e23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:36.587044 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c478b62-2835-4bef-bda2-660e1c0d9e23 None admin] RBAC: Adding query filter params (role.id=64202be313d04a5aab01b866bc558152, scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5, user.id=581c8d476f744f25ad957b35c68a5fe2) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:36.587560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c478b62-2835-4bef-bda2-660e1c0d9e23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:36.593246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c478b62-2835-4bef-bda2-660e1c0d9e23 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:36.599950 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 102/208] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:36 2022] GET /identity/v3/role_assignments?user.id=581c8d476f744f25ad957b35c68a5fe2&scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5&role.id=64202be313d04a5aab01b866bc558152 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:37.181381 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994e078-d88f-4171-a365-9dfa316a0db6 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:37.181788 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994e078-d88f-4171-a365-9dfa316a0db6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:37.181960 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994e078-d88f-4171-a365-9dfa316a0db6 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:37.183623 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 107/209] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:37.189091 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea7cae-2063-45c0-81f7-257304ad4ba1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:37.189660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea7cae-2063-45c0-81f7-257304ad4ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:37.190134 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea7cae-2063-45c0-81f7-257304ad4ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:37.210987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-abea7cae-2063-45c0-81f7-257304ad4ba1 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:37.212494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-abea7cae-2063-45c0-81f7-257304ad4ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:37.287118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362b137-7a1e-4e16-b725-fb9ed8737595 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:37.289168 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362b137-7a1e-4e16-b725-fb9ed8737595 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:37.289168 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362b137-7a1e-4e16-b725-fb9ed8737595 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:37.289746 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 108/210] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:16:37.294796 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a82d17-202b-4b84-b70f-74affff97908 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:37.295314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a82d17-202b-4b84-b70f-74affff97908 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:37.295814 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a82d17-202b-4b84-b70f-74affff97908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:37.303806 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 103/211] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:37.313552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-b6a82d17-202b-4b84-b70f-74affff97908 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:37.314989 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b6a82d17-202b-4b84-b70f-74affff97908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:37.326967 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea7cae-2063-45c0-81f7-257304ad4ba1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:37.386419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 109/212] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:37.391719 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:37.413211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a82d17-202b-4b84-b70f-74affff97908 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:37.428778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:37.436855 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea7cae-2063-45c0-81f7-257304ad4ba1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:37.474211 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4678fc-dfcb-491b-8481-998690e9d631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:37.475831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4678fc-dfcb-491b-8481-998690e9d631 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:37.476349 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4678fc-dfcb-491b-8481-998690e9d631 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:37.476816 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4678fc-dfcb-491b-8481-998690e9d631 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:37.488534 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4678fc-dfcb-491b-8481-998690e9d631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3f4678fc-dfcb-491b-8481-998690e9d631 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:37.496992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:37.497887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:37.498672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:37.498672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:37.498672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:37.498672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:37.498672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:37.498672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:37.498672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:37.498672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:37.498672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:37.498672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:37.498672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:37.498672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:37.501013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 104/213] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:16:37.507333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4678fc-dfcb-491b-8481-998690e9d631 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:37.537158 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a82d17-202b-4b84-b70f-74affff97908 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:37.575892 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71506909-7042-4793-93b3-1cc63576f542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:37.577620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71506909-7042-4793-93b3-1cc63576f542 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:37.578076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71506909-7042-4793-93b3-1cc63576f542 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:37.578782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71506909-7042-4793-93b3-1cc63576f542 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:37.580557 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71506909-7042-4793-93b3-1cc63576f542 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:37.583086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71506909-7042-4793-93b3-1cc63576f542 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:37.587210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 110/214] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:37.592799 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71506909-7042-4793-93b3-1cc63576f542 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:37.643540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4678fc-dfcb-491b-8481-998690e9d631 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:37.657637 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:37.681525 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab67a7-fdfc-445b-b396-ad22618cbd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:37.682522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab67a7-fdfc-445b-b396-ad22618cbd20 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:37.683055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab67a7-fdfc-445b-b396-ad22618cbd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:37.683549 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab67a7-fdfc-445b-b396-ad22618cbd20 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:37.684654 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab67a7-fdfc-445b-b396-ad22618cbd20 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:37.686583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab67a7-fdfc-445b-b396-ad22618cbd20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:37.688454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab67a7-fdfc-445b-b396-ad22618cbd20 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:37.697746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 105/215] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:37.700135 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71506909-7042-4793-93b3-1cc63576f542 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:37.705181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab67a7-fdfc-445b-b396-ad22618cbd20 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:37.734035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:37.743697 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bf1f4-e9c4-49b1-a33a-0f547b99d852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:37.744476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bf1f4-e9c4-49b1-a33a-0f547b99d852 None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:37.744760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bf1f4-e9c4-49b1-a33a-0f547b99d852 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:37.745043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bf1f4-e9c4-49b1-a33a-0f547b99d852 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:37.745916 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bf1f4-e9c4-49b1-a33a-0f547b99d852 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:37.747354 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bf1f4-e9c4-49b1-a33a-0f547b99d852 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:37.776914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 111/216] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:16:37 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:16:37.840102 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab67a7-fdfc-445b-b396-ad22618cbd20 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:37.882636 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a39427-bd4d-4c32-b869-0f789a520210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:37.883730 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a39427-bd4d-4c32-b869-0f789a520210 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:37.884057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a39427-bd4d-4c32-b869-0f789a520210 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:37.884342 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a39427-bd4d-4c32-b869-0f789a520210 None admin] PATH_INFO: `/v3/users/e12f0181026843a5aa4fad5423b6ec2a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:37.901816 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a39427-bd4d-4c32-b869-0f789a520210 None admin] RBAC: Authorizing `identity:get_user(user_id=e12f0181026843a5aa4fad5423b6ec2a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:37.903690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a39427-bd4d-4c32-b869-0f789a520210 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:37.905502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 106/217] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:37 2022] GET /identity/v3/users/e12f0181026843a5aa4fad5423b6ec2a => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:37.911369 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bf1f4-e9c4-49b1-a33a-0f547b99d852 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:38.028668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bf1f4-e9c4-49b1-a33a-0f547b99d852 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:38.043229 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:38.068509 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52b308-1349-42e5-9ce0-699c74e55766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:38.070258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52b308-1349-42e5-9ce0-699c74e55766 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:38.070791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52b308-1349-42e5-9ce0-699c74e55766 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:38.071223 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52b308-1349-42e5-9ce0-699c74e55766 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:38.148076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52b308-1349-42e5-9ce0-699c74e55766 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:38.149645 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52b308-1349-42e5-9ce0-699c74e55766 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:38.266722 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 112/218] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:38.272622 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a39427-bd4d-4c32-b869-0f789a520210 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:38.356568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:38.381599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a39427-bd4d-4c32-b869-0f789a520210 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:38.417595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06464c-dcbc-4cef-9540-6ac179e40eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:38.418467 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06464c-dcbc-4cef-9540-6ac179e40eed None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:38.418752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06464c-dcbc-4cef-9540-6ac179e40eed None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:38.419025 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06464c-dcbc-4cef-9540-6ac179e40eed None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:38.420238 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06464c-dcbc-4cef-9540-6ac179e40eed None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:38.421722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06464c-dcbc-4cef-9540-6ac179e40eed None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:38.423618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 107/219] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:38 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:38.429602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52b308-1349-42e5-9ce0-699c74e55766 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:38.453867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:38.570706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52b308-1349-42e5-9ce0-699c74e55766 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:38.577912 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610ea1f-171c-49de-8227-c858381a72fb None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:38.578773 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610ea1f-171c-49de-8227-c858381a72fb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:38.579245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610ea1f-171c-49de-8227-c858381a72fb None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:38.580915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 108/220] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:16:38.586983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731e0bd-09fa-42f8-bcab-eb12e8d4b326 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:38.587541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731e0bd-09fa-42f8-bcab-eb12e8d4b326 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:38.588013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731e0bd-09fa-42f8-bcab-eb12e8d4b326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:38.606116 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f731e0bd-09fa-42f8-bcab-eb12e8d4b326 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:38.607676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f731e0bd-09fa-42f8-bcab-eb12e8d4b326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:38.617909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83551b2e-a28b-461d-ad78-50152414d871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:38.618581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83551b2e-a28b-461d-ad78-50152414d871 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:38.618869 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83551b2e-a28b-461d-ad78-50152414d871 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:38.619150 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83551b2e-a28b-461d-ad78-50152414d871 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:38.620137 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83551b2e-a28b-461d-ad78-50152414d871 None admin] RBAC: Adding query filter params (role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f, scope.project.id=2bda471206fe4fbb88370df9ab3bdfac, user.id=e12f0181026843a5aa4fad5423b6ec2a) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:38.620456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83551b2e-a28b-461d-ad78-50152414d871 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:38.622078 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83551b2e-a28b-461d-ad78-50152414d871 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:38.629620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 113/221] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:38 2022] GET /identity/v3/role_assignments?user.id=e12f0181026843a5aa4fad5423b6ec2a&scope.project.id=2bda471206fe4fbb88370df9ab3bdfac&role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:38.631131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b319efc-efd4-4f9a-a258-c5d4b66ff5f1 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:38.632171 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b319efc-efd4-4f9a-a258-c5d4b66ff5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:38.632659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b319efc-efd4-4f9a-a258-c5d4b66ff5f1 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:38.634235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 114/222] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:38.646883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba02de-580c-4afa-b663-2c0643177775 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:38.647751 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba02de-580c-4afa-b663-2c0643177775 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:38.648240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba02de-580c-4afa-b663-2c0643177775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:38.681976 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-2dba02de-580c-4afa-b663-2c0643177775 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:38.683609 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2dba02de-580c-4afa-b663-2c0643177775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:38.695122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 109/223] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:38.722128 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731e0bd-09fa-42f8-bcab-eb12e8d4b326 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:38.836521 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:38.844168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731e0bd-09fa-42f8-bcab-eb12e8d4b326 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:38.853014 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 115/224] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:38.877063 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba02de-580c-4afa-b663-2c0643177775 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:38.885713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9925e7-0b5b-4886-9272-d48ad6001508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:38.886751 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9925e7-0b5b-4886-9272-d48ad6001508 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:38.887222 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9925e7-0b5b-4886-9272-d48ad6001508 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:38.887691 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9925e7-0b5b-4886-9272-d48ad6001508 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:38.890354 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:38.901284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9925e7-0b5b-4886-9272-d48ad6001508 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-de9925e7-0b5b-4886-9272-d48ad6001508 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:38.910180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:38.911267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:38.912231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:38.912231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:38.912231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:38.912231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:38.912231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:38.912231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:38.912231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:38.912231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:38.912231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:38.912231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:38.912231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:38.912231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:38.915861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 110/225] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:16:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:16:38.921703 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9925e7-0b5b-4886-9272-d48ad6001508 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:39.032837 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9925e7-0b5b-4886-9272-d48ad6001508 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:39.071171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073721cb-2c95-40d8-9b5c-1fee1bb54a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:39.072101 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073721cb-2c95-40d8-9b5c-1fee1bb54a18 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:39.072539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073721cb-2c95-40d8-9b5c-1fee1bb54a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:39.072964 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073721cb-2c95-40d8-9b5c-1fee1bb54a18 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:39.074013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073721cb-2c95-40d8-9b5c-1fee1bb54a18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:39.074461 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073721cb-2c95-40d8-9b5c-1fee1bb54a18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:39.075978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073721cb-2c95-40d8-9b5c-1fee1bb54a18 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:39.078605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:39.087196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 111/226] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:39.092844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073721cb-2c95-40d8-9b5c-1fee1bb54a18 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:39.218853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba02de-580c-4afa-b663-2c0643177775 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:39.254991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073721cb-2c95-40d8-9b5c-1fee1bb54a18 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:39.296522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac3722-533e-48ab-ad59-967066ed3603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:39.297640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac3722-533e-48ab-ad59-967066ed3603 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:39.298061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac3722-533e-48ab-ad59-967066ed3603 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:39.298473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac3722-533e-48ab-ad59-967066ed3603 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:39.311033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac3722-533e-48ab-ad59-967066ed3603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-84ac3722-533e-48ab-ad59-967066ed3603 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:39.320923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:39.322040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:39.322853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:39.322853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:39.322853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:39.322853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:39.322853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:39.322853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:39.322853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:39.322853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:39.322853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:39.322853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:39.322853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:39.322853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:39.325640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 116/227] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:16:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 449 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:16:39.330860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac3722-533e-48ab-ad59-967066ed3603 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:39.347919 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a3b5e-dffe-4f0e-8cd4-51318cec5985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:39.349105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a3b5e-dffe-4f0e-8cd4-51318cec5985 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:39.349582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a3b5e-dffe-4f0e-8cd4-51318cec5985 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:39.350056 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a3b5e-dffe-4f0e-8cd4-51318cec5985 None admin] PATH_INFO: `/v3/users/f23c7568c8834d65988a7cbd39a5f0d2` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:39.353511 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a3b5e-dffe-4f0e-8cd4-51318cec5985 None admin] RBAC: Authorizing `identity:get_user(user_id=f23c7568c8834d65988a7cbd39a5f0d2)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:39.355251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a3b5e-dffe-4f0e-8cd4-51318cec5985 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:39.358572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 112/228] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:39 2022] GET /identity/v3/users/f23c7568c8834d65988a7cbd39a5f0d2 => generated 283 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:39.374545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a3b5e-dffe-4f0e-8cd4-51318cec5985 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:39.441529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:39.480189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac3722-533e-48ab-ad59-967066ed3603 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:39.529517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:39.551903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83465a69-0262-40b5-95a5-bb6d8884386a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:39.552767 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83465a69-0262-40b5-95a5-bb6d8884386a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:39.552971 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83465a69-0262-40b5-95a5-bb6d8884386a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:39.553377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83465a69-0262-40b5-95a5-bb6d8884386a None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:39.554153 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83465a69-0262-40b5-95a5-bb6d8884386a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:39.554470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83465a69-0262-40b5-95a5-bb6d8884386a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:39.555886 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83465a69-0262-40b5-95a5-bb6d8884386a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:39.573807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a3b5e-dffe-4f0e-8cd4-51318cec5985 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:39.576516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 117/229] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:16:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:39.581826 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83465a69-0262-40b5-95a5-bb6d8884386a None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:39.613413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf864c-b200-404e-b333-02105a011104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:39.615077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf864c-b200-404e-b333-02105a011104 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:39.615554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf864c-b200-404e-b333-02105a011104 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:39.616240 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf864c-b200-404e-b333-02105a011104 None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:39.617788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf864c-b200-404e-b333-02105a011104 None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:39.619720 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf864c-b200-404e-b333-02105a011104 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:39.622120 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 113/230] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:39 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:39.630213 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf864c-b200-404e-b333-02105a011104 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:39.752098 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83465a69-0262-40b5-95a5-bb6d8884386a None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:39.781530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:39.790212 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995617a-191d-4b63-815f-b7dc10d07f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:39.791235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995617a-191d-4b63-815f-b7dc10d07f3c None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:39.791637 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995617a-191d-4b63-815f-b7dc10d07f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:39.792022 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995617a-191d-4b63-815f-b7dc10d07f3c None admin] PATH_INFO: `/v3/users/f23c7568c8834d65988a7cbd39a5f0d2` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:39.793385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995617a-191d-4b63-815f-b7dc10d07f3c None admin] RBAC: Authorizing `identity:get_user(user_id=f23c7568c8834d65988a7cbd39a5f0d2)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:39.794959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995617a-191d-4b63-815f-b7dc10d07f3c None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:39.801159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 118/231] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:39 2022] GET /identity/v3/users/f23c7568c8834d65988a7cbd39a5f0d2 => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:39.803899 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995617a-191d-4b63-815f-b7dc10d07f3c None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:39.853992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf864c-b200-404e-b333-02105a011104 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:39.903206 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:39.910808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccc1da-1cb3-4cfb-8c22-96335118186e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:39.912048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccc1da-1cb3-4cfb-8c22-96335118186e None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:39.913076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccc1da-1cb3-4cfb-8c22-96335118186e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:39.913563 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccc1da-1cb3-4cfb-8c22-96335118186e None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac/users/f23c7568c8834d65988a7cbd39a5f0d2/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:39.916564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ccc1da-1cb3-4cfb-8c22-96335118186e None admin] RBAC: Authorizing `identity:create_grant(project_id=2bda471206fe4fbb88370df9ab3bdfac, user_id=f23c7568c8834d65988a7cbd39a5f0d2, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:39.918563 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ccc1da-1cb3-4cfb-8c22-96335118186e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:39.937499 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 114/232] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:16:39 2022] PUT /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac/users/f23c7568c8834d65988a7cbd39a5f0d2/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:16:40.035062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995617a-191d-4b63-815f-b7dc10d07f3c None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:40.091662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fe0ba-4a86-4e90-a61f-1c1440e3d593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:40.092541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fe0ba-4a86-4e90-a61f-1c1440e3d593 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:40.092822 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fe0ba-4a86-4e90-a61f-1c1440e3d593 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:40.093095 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fe0ba-4a86-4e90-a61f-1c1440e3d593 None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:40.094403 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fe0ba-4a86-4e90-a61f-1c1440e3d593 None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:40.095880 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fe0ba-4a86-4e90-a61f-1c1440e3d593 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:40.097690 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 119/233] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:39 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:40.104405 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccc1da-1cb3-4cfb-8c22-96335118186e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:40.223276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccc1da-1cb3-4cfb-8c22-96335118186e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:40.261301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008c5cc-2442-424f-aa7d-57d3af56bfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:40.262220 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008c5cc-2442-424f-aa7d-57d3af56bfbb None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:40.262508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008c5cc-2442-424f-aa7d-57d3af56bfbb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:40.262788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008c5cc-2442-424f-aa7d-57d3af56bfbb None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac/users/f23c7568c8834d65988a7cbd39a5f0d2/roles/44bd561cc5594ffd9bcc27ef6dad0fc5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:40.265114 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008c5cc-2442-424f-aa7d-57d3af56bfbb None admin] RBAC: Authorizing `identity:create_grant(project_id=2bda471206fe4fbb88370df9ab3bdfac, user_id=f23c7568c8834d65988a7cbd39a5f0d2, role_id=44bd561cc5594ffd9bcc27ef6dad0fc5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:40.266619 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008c5cc-2442-424f-aa7d-57d3af56bfbb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:40.283391 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 115/234] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:16:40 2022] PUT /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac/users/f23c7568c8834d65988a7cbd39a5f0d2/roles/44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:16:40.294882 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e10cc-03a7-4211-bec5-a0c195552b67 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:40.295515 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e10cc-03a7-4211-bec5-a0c195552b67 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:40.295996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e10cc-03a7-4211-bec5-a0c195552b67 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:40.297697 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 120/235] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:40.304366 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b2307-b460-4be4-af45-424d81b6b815 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:40.304733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b2307-b460-4be4-af45-424d81b6b815 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:40.305033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b2307-b460-4be4-af45-424d81b6b815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:40.335377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-882b2307-b460-4be4-af45-424d81b6b815 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:40.336649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-882b2307-b460-4be4-af45-424d81b6b815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:40.398574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:40.425091 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 116/236] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:40.463656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e10cc-03a7-4211-bec5-a0c195552b67 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:40.549804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:40.592839 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e10cc-03a7-4211-bec5-a0c195552b67 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:40.623092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf568b-d574-44e3-b12d-10937851f17f None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:40.623603 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf568b-d574-44e3-b12d-10937851f17f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:40.623706 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf568b-d574-44e3-b12d-10937851f17f None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:40.625221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 117/237] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:16:40.630555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24953c30-b05a-4ed0-96d8-b1b2aba47c55 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:40.631744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24953c30-b05a-4ed0-96d8-b1b2aba47c55 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:40.632350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24953c30-b05a-4ed0-96d8-b1b2aba47c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:40.644333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e12f0e-8dae-4080-ace6-109f8a2567c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:40.645435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e12f0e-8dae-4080-ace6-109f8a2567c3 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:40.647845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e12f0e-8dae-4080-ace6-109f8a2567c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:40.648322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e12f0e-8dae-4080-ace6-109f8a2567c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:40.649819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e12f0e-8dae-4080-ace6-109f8a2567c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:40.651487 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e12f0e-8dae-4080-ace6-109f8a2567c3 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:40.653652 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 121/238] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:40.664788 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e12f0e-8dae-4080-ace6-109f8a2567c3 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:40.664941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-24953c30-b05a-4ed0-96d8-b1b2aba47c55 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:40.666160 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-24953c30-b05a-4ed0-96d8-b1b2aba47c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:40.807045 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 118/239] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:40.843988 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e12f0e-8dae-4080-ace6-109f8a2567c3 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:40.845731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24953c30-b05a-4ed0-96d8-b1b2aba47c55 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:40.925088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:40.937755 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878eaff5-4cb9-42d9-a620-cc6f0b6e91d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:40.938390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878eaff5-4cb9-42d9-a620-cc6f0b6e91d2 None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:40.938677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878eaff5-4cb9-42d9-a620-cc6f0b6e91d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:40.938955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878eaff5-4cb9-42d9-a620-cc6f0b6e91d2 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:40.939683 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878eaff5-4cb9-42d9-a620-cc6f0b6e91d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:40.941149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878eaff5-4cb9-42d9-a620-cc6f0b6e91d2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:40.983454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 122/240] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:16:40 2022] POST /identity/v3/users => generated 293 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:16:41.001758 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24953c30-b05a-4ed0-96d8-b1b2aba47c55 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:41.011369 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a9539-1635-448a-904d-f46d948dbf00 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:41.011369 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a9539-1635-448a-904d-f46d948dbf00 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:41.011705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a9539-1635-448a-904d-f46d948dbf00 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:41.013490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 123/241] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:16:41.018984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04118ad0-9f23-4a13-b7dc-e6f0baee3a16 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:41.019274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04118ad0-9f23-4a13-b7dc-e6f0baee3a16 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:41.019553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04118ad0-9f23-4a13-b7dc-e6f0baee3a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:41.025793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:41.040324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-04118ad0-9f23-4a13-b7dc-e6f0baee3a16 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:41.045173 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-04118ad0-9f23-4a13-b7dc-e6f0baee3a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:41.055418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3f418-b52c-4453-8a83-f47f6f949723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:41.056606 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3f418-b52c-4453-8a83-f47f6f949723 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:41.057219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3f418-b52c-4453-8a83-f47f6f949723 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:41.057762 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3f418-b52c-4453-8a83-f47f6f949723 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:41.080146 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3f418-b52c-4453-8a83-f47f6f949723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-5db3f418-b52c-4453-8a83-f47f6f949723 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:41.089124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:41.090090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:41.090873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:41.090873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:41.090873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:41.090873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:41.090873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:41.090873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:41.090873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:41.090873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:41.090873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:41.090873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:41.090873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:41.090873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:41.090873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 119/242] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:16:41.096744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3f418-b52c-4453-8a83-f47f6f949723 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:41.139688 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 124/243] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:41.165247 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04118ad0-9f23-4a13-b7dc-e6f0baee3a16 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:41.282284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3f418-b52c-4453-8a83-f47f6f949723 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:41.292032 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04118ad0-9f23-4a13-b7dc-e6f0baee3a16 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:41.314924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:41.323679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf11754-1be0-4433-b2b2-863d49788db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:41.324771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf11754-1be0-4433-b2b2-863d49788db4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:41.325731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf11754-1be0-4433-b2b2-863d49788db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:41.326588 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf11754-1be0-4433-b2b2-863d49788db4 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:41.327596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf11754-1be0-4433-b2b2-863d49788db4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:41.328161 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf11754-1be0-4433-b2b2-863d49788db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:41.330586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf11754-1be0-4433-b2b2-863d49788db4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:41.335103 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fd1d8-d59e-46ae-bb94-789be6fbac4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:41.335902 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fd1d8-d59e-46ae-bb94-789be6fbac4c None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:41.336189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fd1d8-d59e-46ae-bb94-789be6fbac4c None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:41.336466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fd1d8-d59e-46ae-bb94-789be6fbac4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:41.341200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 120/244] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:41.346824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf11754-1be0-4433-b2b2-863d49788db4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:41.348844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fd1d8-d59e-46ae-bb94-789be6fbac4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-f40fd1d8-d59e-46ae-bb94-789be6fbac4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:41.360002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:41.360912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:41.361760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:41.361760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:41.361760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:41.361760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:41.361760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:41.361760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:41.361760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:41.361760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:41.361760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:41.361760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:41.361760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:41.361760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:41.361760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 125/245] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:16:41.367402 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fd1d8-d59e-46ae-bb94-789be6fbac4c None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:41.399899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:41.505001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf11754-1be0-4433-b2b2-863d49788db4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:41.524648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fd1d8-d59e-46ae-bb94-789be6fbac4c None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:41.560912 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c72a5-f7a9-4ae6-bde3-0ac40b7ea6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:41.561793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c72a5-f7a9-4ae6-bde3-0ac40b7ea6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:41.562077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c72a5-f7a9-4ae6-bde3-0ac40b7ea6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:41.562353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c72a5-f7a9-4ae6-bde3-0ac40b7ea6f2 None admin] PATH_INFO: `/v3/users/44610ee1059c4d9789ba7e3d9f8542e7` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:41.578191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1362c-71cc-4d78-af21-91144b196488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:41.579338 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1362c-71cc-4d78-af21-91144b196488 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:41.579836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1362c-71cc-4d78-af21-91144b196488 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:41.580310 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1362c-71cc-4d78-af21-91144b196488 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:41.581416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c72a5-f7a9-4ae6-bde3-0ac40b7ea6f2 None admin] RBAC: Authorizing `identity:get_user(user_id=44610ee1059c4d9789ba7e3d9f8542e7)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:41.581860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1362c-71cc-4d78-af21-91144b196488 None admin] RBAC: Adding query filter params (name=member) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:41.582341 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1362c-71cc-4d78-af21-91144b196488 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:41.582890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c72a5-f7a9-4ae6-bde3-0ac40b7ea6f2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:41.584140 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1362c-71cc-4d78-af21-91144b196488 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:41.592573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 121/246] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:41 2022] GET /identity/v3/users/44610ee1059c4d9789ba7e3d9f8542e7 => generated 297 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:41.593860 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c72a5-f7a9-4ae6-bde3-0ac40b7ea6f2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:41.595146 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 126/247] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:41.601615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1362c-71cc-4d78-af21-91144b196488 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:41.651955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:41.700954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:41.712935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c72a5-f7a9-4ae6-bde3-0ac40b7ea6f2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:41.723062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1362c-71cc-4d78-af21-91144b196488 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:41.750627 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a9adc-e4cf-4811-a666-c2334c7ada1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:41.751218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a9adc-e4cf-4811-a666-c2334c7ada1c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:41.751505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a9adc-e4cf-4811-a666-c2334c7ada1c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:41.751782 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a9adc-e4cf-4811-a666-c2334c7ada1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:41.762166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ce127-370b-45f9-b7ac-969baaeda88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:41.763429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ce127-370b-45f9-b7ac-969baaeda88f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:41.763909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ce127-370b-45f9-b7ac-969baaeda88f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:41.764465 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ce127-370b-45f9-b7ac-969baaeda88f None admin] PATH_INFO: `/v3/users/e12f0181026843a5aa4fad5423b6ec2a` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:41.765888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ce127-370b-45f9-b7ac-969baaeda88f None admin] RBAC: Authorizing `identity:get_user(user_id=e12f0181026843a5aa4fad5423b6ec2a)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:41.767704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ce127-370b-45f9-b7ac-969baaeda88f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:41.776394 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 127/248] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:41 2022] GET /identity/v3/users/e12f0181026843a5aa4fad5423b6ec2a => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:41.777904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ce127-370b-45f9-b7ac-969baaeda88f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:41.833350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a9adc-e4cf-4811-a666-c2334c7ada1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:41.835441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a9adc-e4cf-4811-a666-c2334c7ada1c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:41.892308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ce127-370b-45f9-b7ac-969baaeda88f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:41.903150 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:41.934182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf1866-0e84-4be1-a368-650a6f6f4fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:41.935751 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf1866-0e84-4be1-a368-650a6f6f4fba None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:41.936458 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf1866-0e84-4be1-a368-650a6f6f4fba None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:41.937418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:41.937904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf1866-0e84-4be1-a368-650a6f6f4fba None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:41.940185 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf1866-0e84-4be1-a368-650a6f6f4fba None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:41.942637 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf1866-0e84-4be1-a368-650a6f6f4fba None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:41.946577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 128/249] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:41 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:41.949197 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 122/250] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 15:16:41.952654 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf1866-0e84-4be1-a368-650a6f6f4fba None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:41.954488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a9adc-e4cf-4811-a666-c2334c7ada1c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:42.064402 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf1866-0e84-4be1-a368-650a6f6f4fba None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:42.106475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28288c80-08a2-4a69-b5bc-76367a911d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:42.107807 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28288c80-08a2-4a69-b5bc-76367a911d74 None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:42.108357 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28288c80-08a2-4a69-b5bc-76367a911d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:42.108830 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28288c80-08a2-4a69-b5bc-76367a911d74 None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac/users/e12f0181026843a5aa4fad5423b6ec2a/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:42.111310 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28288c80-08a2-4a69-b5bc-76367a911d74 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bda471206fe4fbb88370df9ab3bdfac, user_id=e12f0181026843a5aa4fad5423b6ec2a, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:42.113011 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28288c80-08a2-4a69-b5bc-76367a911d74 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:42.118666 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a9adc-e4cf-4811-a666-c2334c7ada1c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:42.127979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 129/251] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:16:41 2022] PUT /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac/users/e12f0181026843a5aa4fad5423b6ec2a/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:16:42.176946 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2b59b-1dd1-411c-849c-3af52d74923b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:42.177859 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2b59b-1dd1-411c-849c-3af52d74923b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:42.178775 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2b59b-1dd1-411c-849c-3af52d74923b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:42.179058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2b59b-1dd1-411c-849c-3af52d74923b None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:42.181527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2b59b-1dd1-411c-849c-3af52d74923b None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:42.183057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2b59b-1dd1-411c-849c-3af52d74923b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:42.185227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 123/252] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:41 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:42.191421 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28288c80-08a2-4a69-b5bc-76367a911d74 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:42.276263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:42.356033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28288c80-08a2-4a69-b5bc-76367a911d74 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:42.402320 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32df64-a81a-44e5-aa06-b2e8f8aa9a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:42.403153 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32df64-a81a-44e5-aa06-b2e8f8aa9a76 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:42.403437 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32df64-a81a-44e5-aa06-b2e8f8aa9a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:42.403713 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32df64-a81a-44e5-aa06-b2e8f8aa9a76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:42.404565 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32df64-a81a-44e5-aa06-b2e8f8aa9a76 None admin] RBAC: Adding query filter params (role.id=64202be313d04a5aab01b866bc558152, scope.project.id=2bda471206fe4fbb88370df9ab3bdfac, user.id=44610ee1059c4d9789ba7e3d9f8542e7) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:42.404883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32df64-a81a-44e5-aa06-b2e8f8aa9a76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:42.406380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32df64-a81a-44e5-aa06-b2e8f8aa9a76 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:42.414427 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 130/253] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:42 2022] GET /identity/v3/role_assignments?user.id=44610ee1059c4d9789ba7e3d9f8542e7&scope.project.id=2bda471206fe4fbb88370df9ab3bdfac&role.id=64202be313d04a5aab01b866bc558152 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:42.749913 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e64629-44a5-45b6-af51-3074eaea5cf4 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:42.750425 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e64629-44a5-45b6-af51-3074eaea5cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:42.750501 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e64629-44a5-45b6-af51-3074eaea5cf4 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:42.752125 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 124/254] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:42.757285 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bcefe-e894-49b0-8df9-e19d7756b898 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:42.757708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bcefe-e894-49b0-8df9-e19d7756b898 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:42.757894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bcefe-e894-49b0-8df9-e19d7756b898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:42.778386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-fa4bcefe-e894-49b0-8df9-e19d7756b898 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:42.779578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-fa4bcefe-e894-49b0-8df9-e19d7756b898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:42.860000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 131/255] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:42.899683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e64629-44a5-45b6-af51-3074eaea5cf4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:42.977345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:43.025778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e64629-44a5-45b6-af51-3074eaea5cf4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:43.061364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c52c5-8fde-420f-9415-a3028fd82169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:43.062166 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c52c5-8fde-420f-9415-a3028fd82169 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:43.062445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c52c5-8fde-420f-9415-a3028fd82169 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:43.062717 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c52c5-8fde-420f-9415-a3028fd82169 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:43.075407 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c52c5-8fde-420f-9415-a3028fd82169 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-708c52c5-8fde-420f-9415-a3028fd82169 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:43.083353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:43.084867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:43.085736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:43.085736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:43.085736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:43.085736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:43.085736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:43.085736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:43.085736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:43.085736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:43.085736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:43.085736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:43.085736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:43.085736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:43.085736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 125/256] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:16:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:43.094485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bcefe-e894-49b0-8df9-e19d7756b898 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:43.139437 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:43.234126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bcefe-e894-49b0-8df9-e19d7756b898 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:43.275754 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280137e3-585f-4797-8db3-de7ca8a7c2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:43.276488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280137e3-585f-4797-8db3-de7ca8a7c2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:43.276721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280137e3-585f-4797-8db3-de7ca8a7c2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:43.277003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280137e3-585f-4797-8db3-de7ca8a7c2f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:43.278125 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280137e3-585f-4797-8db3-de7ca8a7c2f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:43.278439 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280137e3-585f-4797-8db3-de7ca8a7c2f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:43.279837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280137e3-585f-4797-8db3-de7ca8a7c2f0 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:43.288728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 132/257] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:43.294312 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c52c5-8fde-420f-9415-a3028fd82169 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:43.377130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab9647-ec9f-411f-a123-3c45e83ed0ea None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:43.377507 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab9647-ec9f-411f-a123-3c45e83ed0ea None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:43.377860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab9647-ec9f-411f-a123-3c45e83ed0ea None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:43.379385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 133/258] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:16:43.395589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedf82c-e775-4cd9-84e5-62cdf59464f1 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:43.395883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedf82c-e775-4cd9-84e5-62cdf59464f1 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:43.396169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedf82c-e775-4cd9-84e5-62cdf59464f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:43.426279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-0bedf82c-e775-4cd9-84e5-62cdf59464f1 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:43.427579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0bedf82c-e775-4cd9-84e5-62cdf59464f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:43.444931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c52c5-8fde-420f-9415-a3028fd82169 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:43.465915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:43.484699 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f01625-ca75-4550-b043-7fc6248622e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:43.485616 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f01625-ca75-4550-b043-7fc6248622e4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:43.485901 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f01625-ca75-4550-b043-7fc6248622e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:43.486184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f01625-ca75-4550-b043-7fc6248622e4 None admin] PATH_INFO: `/v3/users/f23c7568c8834d65988a7cbd39a5f0d2` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:43.487386 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f01625-ca75-4550-b043-7fc6248622e4 None admin] RBAC: Authorizing `identity:get_user(user_id=f23c7568c8834d65988a7cbd39a5f0d2)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:43.488853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f01625-ca75-4550-b043-7fc6248622e4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:43.491139 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 126/259] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:43 2022] GET /identity/v3/users/f23c7568c8834d65988a7cbd39a5f0d2 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:43.496626 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f01625-ca75-4550-b043-7fc6248622e4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:43.575199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 134/260] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:43.614855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bedf82c-e775-4cd9-84e5-62cdf59464f1 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:43.654121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:43.670097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f01625-ca75-4550-b043-7fc6248622e4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:43.739012 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9d2bd-b2a5-4319-83f5-c8c447666520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:43.739681 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9d2bd-b2a5-4319-83f5-c8c447666520 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:43.739965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9d2bd-b2a5-4319-83f5-c8c447666520 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:43.740246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9d2bd-b2a5-4319-83f5-c8c447666520 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:43.814136 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bedf82c-e775-4cd9-84e5-62cdf59464f1 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:43.849705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:43.856628 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9d2bd-b2a5-4319-83f5-c8c447666520 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:43.858135 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9d2bd-b2a5-4319-83f5-c8c447666520 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:43.906403 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8316a-dba8-44a2-9a5d-a1a2706dad0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:43.907227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8316a-dba8-44a2-9a5d-a1a2706dad0e None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:43.907503 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8316a-dba8-44a2-9a5d-a1a2706dad0e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:43.907767 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8316a-dba8-44a2-9a5d-a1a2706dad0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:43.941801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8316a-dba8-44a2-9a5d-a1a2706dad0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-cdf8316a-dba8-44a2-9a5d-a1a2706dad0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:43.950392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:43.952530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:43.954621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:43.954621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:43.954621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:43.954621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:43.954621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:43.954621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:43.954621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:43.954621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:43.954621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:43.954621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:43.954621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:43.954621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:43.954621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 135/261] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:16:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:43.966227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8316a-dba8-44a2-9a5d-a1a2706dad0e None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:44.017256 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 127/262] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 15:16:44.019071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9d2bd-b2a5-4319-83f5-c8c447666520 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:44.121991 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:44.141741 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:44.163411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8316a-dba8-44a2-9a5d-a1a2706dad0e None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:44.166609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9d2bd-b2a5-4319-83f5-c8c447666520 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:44.237909 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dce12b-f5e1-41f8-ad87-e7a46e105c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:44.238431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8aa62c-8985-42c1-9e24-0322608fb94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:44.238798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dce12b-f5e1-41f8-ad87-e7a46e105c4d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:44.239077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dce12b-f5e1-41f8-ad87-e7a46e105c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:44.239178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8aa62c-8985-42c1-9e24-0322608fb94d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:44.239346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dce12b-f5e1-41f8-ad87-e7a46e105c4d None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:44.239442 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8aa62c-8985-42c1-9e24-0322608fb94d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:44.239655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8aa62c-8985-42c1-9e24-0322608fb94d None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:44.240516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8aa62c-8985-42c1-9e24-0322608fb94d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:44.240831 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8aa62c-8985-42c1-9e24-0322608fb94d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:44.240931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dce12b-f5e1-41f8-ad87-e7a46e105c4d None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:44.242325 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dce12b-f5e1-41f8-ad87-e7a46e105c4d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:44.242430 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8aa62c-8985-42c1-9e24-0322608fb94d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:44.248569 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 128/263] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:44 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:44.249677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dce12b-f5e1-41f8-ad87-e7a46e105c4d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:44.263609 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 136/264] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:16:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:44.268924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8aa62c-8985-42c1-9e24-0322608fb94d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:44.372942 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dce12b-f5e1-41f8-ad87-e7a46e105c4d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:44.418781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe69542-b156-4e1d-863e-04dd2902b72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:44.419455 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe69542-b156-4e1d-863e-04dd2902b72f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:44.419735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe69542-b156-4e1d-863e-04dd2902b72f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:44.420007 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe69542-b156-4e1d-863e-04dd2902b72f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:44.420851 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe69542-b156-4e1d-863e-04dd2902b72f None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=2bda471206fe4fbb88370df9ab3bdfac, user.id=f23c7568c8834d65988a7cbd39a5f0d2) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:44.421174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe69542-b156-4e1d-863e-04dd2902b72f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:44.422622 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe69542-b156-4e1d-863e-04dd2902b72f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:44.435617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 129/265] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:44 2022] GET /identity/v3/role_assignments?user.id=f23c7568c8834d65988a7cbd39a5f0d2&scope.project.id=2bda471206fe4fbb88370df9ab3bdfac&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:44.444242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cde64d-3788-41a7-a80c-71cabd6b1608 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:44.444524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cde64d-3788-41a7-a80c-71cabd6b1608 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:44.444965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cde64d-3788-41a7-a80c-71cabd6b1608 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:44.446525 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 130/266] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:16:44.452402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dbcaf-8890-4e6b-8545-9e06b73e52c3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:44.452681 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dbcaf-8890-4e6b-8545-9e06b73e52c3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:44.453119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dbcaf-8890-4e6b-8545-9e06b73e52c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:44.494100 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1a0dbcaf-8890-4e6b-8545-9e06b73e52c3 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:44.495680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1a0dbcaf-8890-4e6b-8545-9e06b73e52c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:44.520849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:44.563228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8aa62c-8985-42c1-9e24-0322608fb94d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:44.576510 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 131/267] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:44.603482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dbcaf-8890-4e6b-8545-9e06b73e52c3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:44.656971 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389c5e4-8a47-41f0-868c-c28d103c10ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:44.658241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389c5e4-8a47-41f0-868c-c28d103c10ad None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:44.658849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389c5e4-8a47-41f0-868c-c28d103c10ad None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:44.659328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389c5e4-8a47-41f0-868c-c28d103c10ad None admin] PATH_INFO: `/v3/users/f23c7568c8834d65988a7cbd39a5f0d2` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:44.663460 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389c5e4-8a47-41f0-868c-c28d103c10ad None admin] RBAC: Authorizing `identity:get_user(user_id=f23c7568c8834d65988a7cbd39a5f0d2)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:44.665602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389c5e4-8a47-41f0-868c-c28d103c10ad None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:44.671628 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 137/268] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:44 2022] GET /identity/v3/users/f23c7568c8834d65988a7cbd39a5f0d2 => generated 283 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:44.679197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389c5e4-8a47-41f0-868c-c28d103c10ad None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:44.704381 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:44.781585 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dbcaf-8890-4e6b-8545-9e06b73e52c3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:44.799392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389c5e4-8a47-41f0-868c-c28d103c10ad None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:44.820011 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d4663-f760-43e8-a22b-2341e15c2865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:44.820826 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d4663-f760-43e8-a22b-2341e15c2865 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:44.821113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d4663-f760-43e8-a22b-2341e15c2865 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:44.821421 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d4663-f760-43e8-a22b-2341e15c2865 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:44.832693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d4663-f760-43e8-a22b-2341e15c2865 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:44.839120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6217270-68af-43f6-8ba6-63891b73953e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:44.840378 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6217270-68af-43f6-8ba6-63891b73953e None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:44.840863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6217270-68af-43f6-8ba6-63891b73953e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-0f2d4663-f760-43e8-a22b-2341e15c2865 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:44.841077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:44.841960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:44.842755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:44.842755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:44.842755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:44.842755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:44.842755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:44.842755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:44.842755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:44.842755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:44.842755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:44.842755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:44.842755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:44.842755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:44.842755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 132/269] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:16:44.843903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a09a95-324f-4874-9783-ba0c40ae57b3 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:44.844194 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a09a95-324f-4874-9783-ba0c40ae57b3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:44.844479 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a09a95-324f-4874-9783-ba0c40ae57b3 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:44.844951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6217270-68af-43f6-8ba6-63891b73953e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:44.849387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 133/270] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:44.852671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a09a95-324f-4874-9783-ba0c40ae57b3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:44.872552 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:44.938162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6217270-68af-43f6-8ba6-63891b73953e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:44.940152 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6217270-68af-43f6-8ba6-63891b73953e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:44.956405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:44.982652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a09a95-324f-4874-9783-ba0c40ae57b3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:45.020136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca73fad-98de-4276-9e5c-b044b58e24ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:45.021060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca73fad-98de-4276-9e5c-b044b58e24ca None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:45.021474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca73fad-98de-4276-9e5c-b044b58e24ca None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:45.021937 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca73fad-98de-4276-9e5c-b044b58e24ca None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:45.023052 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca73fad-98de-4276-9e5c-b044b58e24ca None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:45.023318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca73fad-98de-4276-9e5c-b044b58e24ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:45.024818 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca73fad-98de-4276-9e5c-b044b58e24ca None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:45.034307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 134/271] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:45.035802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1d3d6-5c09-4921-95aa-af7239a20ec9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:45.041034 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1d3d6-5c09-4921-95aa-af7239a20ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:45.041034 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1d3d6-5c09-4921-95aa-af7239a20ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:45.058693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-53b1d3d6-5c09-4921-95aa-af7239a20ec9 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:45.060282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-53b1d3d6-5c09-4921-95aa-af7239a20ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:45.063473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 138/272] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:45.064514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6217270-68af-43f6-8ba6-63891b73953e None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:45.136238 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 135/273] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:45.145804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1d3d6-5c09-4921-95aa-af7239a20ec9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:45.148719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:45.176472 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6217270-68af-43f6-8ba6-63891b73953e None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:45.196024 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:45.215663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f773d-3f54-4c8e-bad0-5434b668b443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:45.216894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f773d-3f54-4c8e-bad0-5434b668b443 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:45.217419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f773d-3f54-4c8e-bad0-5434b668b443 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:45.218030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f773d-3f54-4c8e-bad0-5434b668b443 None admin] PATH_INFO: `/v3/users/01bafcfad5c44384b68490bb40abc8b5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:45.234438 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f773d-3f54-4c8e-bad0-5434b668b443 None admin] RBAC: Authorizing `identity:get_user(user_id=01bafcfad5c44384b68490bb40abc8b5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:45.236434 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f773d-3f54-4c8e-bad0-5434b668b443 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:45.243716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 139/274] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:45 2022] GET /identity/v3/users/01bafcfad5c44384b68490bb40abc8b5 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:45.254983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f773d-3f54-4c8e-bad0-5434b668b443 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:45.279337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1d3d6-5c09-4921-95aa-af7239a20ec9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:45.326044 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695ce89-0f01-439e-b0d7-ff3843f36373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:45.327324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695ce89-0f01-439e-b0d7-ff3843f36373 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:45.327850 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695ce89-0f01-439e-b0d7-ff3843f36373 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:45.328143 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695ce89-0f01-439e-b0d7-ff3843f36373 None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:45.329398 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695ce89-0f01-439e-b0d7-ff3843f36373 None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:45.331107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695ce89-0f01-439e-b0d7-ff3843f36373 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:45.333469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 136/275] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:45 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:45.335123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695ce89-0f01-439e-b0d7-ff3843f36373 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:45.370972 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f773d-3f54-4c8e-bad0-5434b668b443 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:45.407563 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9da18-02ba-4de5-bc7b-0db21b0f7e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:45.408751 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9da18-02ba-4de5-bc7b-0db21b0f7e39 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:45.409278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9da18-02ba-4de5-bc7b-0db21b0f7e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:45.409759 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9da18-02ba-4de5-bc7b-0db21b0f7e39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:45.411186 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9da18-02ba-4de5-bc7b-0db21b0f7e39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:45.411958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:45.413167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9da18-02ba-4de5-bc7b-0db21b0f7e39 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:45.419382 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 140/276] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:45.419382 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9da18-02ba-4de5-bc7b-0db21b0f7e39 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:45.450151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695ce89-0f01-439e-b0d7-ff3843f36373 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:45.465223 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:45.486246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44730b6-44b5-4b63-8813-d0fbe303f2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:45.486899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44730b6-44b5-4b63-8813-d0fbe303f2aa None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:45.487178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44730b6-44b5-4b63-8813-d0fbe303f2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:45.487452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44730b6-44b5-4b63-8813-d0fbe303f2aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:45.488285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44730b6-44b5-4b63-8813-d0fbe303f2aa None admin] RBAC: Adding query filter params (role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f, user.id=01bafcfad5c44384b68490bb40abc8b5, scope.system=all) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:45.488597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44730b6-44b5-4b63-8813-d0fbe303f2aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:45.490060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44730b6-44b5-4b63-8813-d0fbe303f2aa None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:45.505182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 137/277] 10.222.0.42 () {56 vars in 1308 bytes} [Tue Jul 19 15:16:45 2022] GET /identity/v3/role_assignments?user.id=01bafcfad5c44384b68490bb40abc8b5&scope.system=all&role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:45.505182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad56b6b-65fb-4afc-b401-78b2025de37b None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:45.505182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad56b6b-65fb-4afc-b401-78b2025de37b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:45.505182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad56b6b-65fb-4afc-b401-78b2025de37b None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:45.505182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 138/278] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:45.505182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad56b6b-65fb-4afc-b401-78b2025de37b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:45.535116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9da18-02ba-4de5-bc7b-0db21b0f7e39 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:45.572866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4ccc6-2c89-491c-aa86-b4a746ba66f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:45.573858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4ccc6-2c89-491c-aa86-b4a746ba66f6 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:45.573858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4ccc6-2c89-491c-aa86-b4a746ba66f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:45.574130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4ccc6-2c89-491c-aa86-b4a746ba66f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:45.574992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4ccc6-2c89-491c-aa86-b4a746ba66f6 None admin] RBAC: Adding query filter params (role.id=44bd561cc5594ffd9bcc27ef6dad0fc5, scope.project.id=2bda471206fe4fbb88370df9ab3bdfac, user.id=f23c7568c8834d65988a7cbd39a5f0d2) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:45.575312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4ccc6-2c89-491c-aa86-b4a746ba66f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:45.576804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4ccc6-2c89-491c-aa86-b4a746ba66f6 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:45.590914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 141/279] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:45 2022] GET /identity/v3/role_assignments?user.id=f23c7568c8834d65988a7cbd39a5f0d2&scope.project.id=2bda471206fe4fbb88370df9ab3bdfac&role.id=44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:45.590914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089418c9-680f-464d-8840-7ba9adbd1157 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:45.590914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089418c9-680f-464d-8840-7ba9adbd1157 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:45.590914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089418c9-680f-464d-8840-7ba9adbd1157 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:45.590914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 142/280] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:45.590914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f743d8-6838-4afa-8c1d-f077cefb81f6 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:45.591479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f743d8-6838-4afa-8c1d-f077cefb81f6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:45.591479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f743d8-6838-4afa-8c1d-f077cefb81f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:45.611232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-32f743d8-6838-4afa-8c1d-f077cefb81f6 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:45.612738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-32f743d8-6838-4afa-8c1d-f077cefb81f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:45.636226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad56b6b-65fb-4afc-b401-78b2025de37b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:45.688586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd71786-6fbd-44c9-b1d7-e7a269b5e1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:45.689685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd71786-6fbd-44c9-b1d7-e7a269b5e1b5 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:45.690034 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd71786-6fbd-44c9-b1d7-e7a269b5e1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:45.690843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd71786-6fbd-44c9-b1d7-e7a269b5e1b5 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:45.691977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd71786-6fbd-44c9-b1d7-e7a269b5e1b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:45.695229 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd71786-6fbd-44c9-b1d7-e7a269b5e1b5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:45.702905 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:45.707653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 143/281] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:45.710159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae016454-0e25-46b8-9ae4-9fa9ecbde806 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:45.710526 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae016454-0e25-46b8-9ae4-9fa9ecbde806 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:45.710855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae016454-0e25-46b8-9ae4-9fa9ecbde806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:45.735495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 139/282] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:16:45 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:16:45.737037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd71786-6fbd-44c9-b1d7-e7a269b5e1b5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:45.737624 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ae016454-0e25-46b8-9ae4-9fa9ecbde806 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:45.738915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ae016454-0e25-46b8-9ae4-9fa9ecbde806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:45.782276 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:45.849165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 144/283] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:45.869231 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae016454-0e25-46b8-9ae4-9fa9ecbde806 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:45.895688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd71786-6fbd-44c9-b1d7-e7a269b5e1b5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:45.953873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e49dd-bb2d-497c-baf9-444b5a62ce8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:45.955560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e49dd-bb2d-497c-baf9-444b5a62ce8e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:45.956305 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e49dd-bb2d-497c-baf9-444b5a62ce8e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:45.956758 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e49dd-bb2d-497c-baf9-444b5a62ce8e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:45.975539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e49dd-bb2d-497c-baf9-444b5a62ce8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f33e49dd-bb2d-497c-baf9-444b5a62ce8e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:45.984543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:45.985491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:45.986313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:45.986313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:45.986313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:45.986313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:45.986313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:45.986313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:45.986313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:45.986313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:45.986313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:45.986313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:45.986313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:45.986313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:45.986313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 140/284] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:45.995849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e49dd-bb2d-497c-baf9-444b5a62ce8e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:46.021381 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae016454-0e25-46b8-9ae4-9fa9ecbde806 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:46.068053 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:46.068934 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a56a71-c9a6-4989-85d7-d61a842318d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:46.069849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a56a71-c9a6-4989-85d7-d61a842318d0 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:46.070138 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a56a71-c9a6-4989-85d7-d61a842318d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:46.070419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a56a71-c9a6-4989-85d7-d61a842318d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:46.087684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a56a71-c9a6-4989-85d7-d61a842318d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-60a56a71-c9a6-4989-85d7-d61a842318d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:46.096082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:46.097278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:46.098387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:46.098387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:46.098387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:46.098387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:46.098387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:46.098387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:46.098387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:46.098387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:46.098387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:46.098387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:46.098387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:46.098387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:46.098387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 145/285] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:16:46.106745 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a56a71-c9a6-4989-85d7-d61a842318d0 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:46.134518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:46.143250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e49dd-bb2d-497c-baf9-444b5a62ce8e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:46.190524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c5439-3fe9-4b60-bb96-d5677eda955b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:46.191214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c5439-3fe9-4b60-bb96-d5677eda955b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:46.191503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c5439-3fe9-4b60-bb96-d5677eda955b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:46.191783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c5439-3fe9-4b60-bb96-d5677eda955b None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:46.192558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c5439-3fe9-4b60-bb96-d5677eda955b None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:46.192873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c5439-3fe9-4b60-bb96-d5677eda955b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:46.194422 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c5439-3fe9-4b60-bb96-d5677eda955b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:46.207000 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 141/286] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:46.214614 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c5439-3fe9-4b60-bb96-d5677eda955b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:46.234543 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a56a71-c9a6-4989-85d7-d61a842318d0 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:46.275807 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc444c76-353c-43db-ba4d-6d9110640052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:46.276774 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc444c76-353c-43db-ba4d-6d9110640052 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:46.277288 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc444c76-353c-43db-ba4d-6d9110640052 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:46.278042 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc444c76-353c-43db-ba4d-6d9110640052 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:46.279093 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc444c76-353c-43db-ba4d-6d9110640052 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:46.279583 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc444c76-353c-43db-ba4d-6d9110640052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:46.281272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc444c76-353c-43db-ba4d-6d9110640052 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:46.291591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 146/287] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:46.298377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc444c76-353c-43db-ba4d-6d9110640052 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:46.339156 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c5439-3fe9-4b60-bb96-d5677eda955b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:46.366087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:46.378877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d89a63-db85-4eee-a762-9ac6b89e8742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:46.379765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d89a63-db85-4eee-a762-9ac6b89e8742 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:46.380059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d89a63-db85-4eee-a762-9ac6b89e8742 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:46.380345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d89a63-db85-4eee-a762-9ac6b89e8742 None admin] PATH_INFO: `/v3/users/e12f0181026843a5aa4fad5423b6ec2a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:46.382197 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d89a63-db85-4eee-a762-9ac6b89e8742 None admin] RBAC: Authorizing `identity:get_user(user_id=e12f0181026843a5aa4fad5423b6ec2a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:46.383842 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d89a63-db85-4eee-a762-9ac6b89e8742 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:46.387117 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 142/288] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:46 2022] GET /identity/v3/users/e12f0181026843a5aa4fad5423b6ec2a => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:46.393395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d89a63-db85-4eee-a762-9ac6b89e8742 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:46.395447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:46.418861 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc444c76-353c-43db-ba4d-6d9110640052 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:46.483983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b8f60-76c7-4d86-b068-650193c46b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:46.484863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b8f60-76c7-4d86-b068-650193c46b28 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:46.485157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b8f60-76c7-4d86-b068-650193c46b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:46.485433 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b8f60-76c7-4d86-b068-650193c46b28 None admin] PATH_INFO: `/v3/users/44610ee1059c4d9789ba7e3d9f8542e7` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:46.487858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b8f60-76c7-4d86-b068-650193c46b28 None admin] RBAC: Authorizing `identity:get_user(user_id=44610ee1059c4d9789ba7e3d9f8542e7)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:46.489336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b8f60-76c7-4d86-b068-650193c46b28 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:46.491361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 147/289] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:46 2022] GET /identity/v3/users/44610ee1059c4d9789ba7e3d9f8542e7 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:46.499700 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b8f60-76c7-4d86-b068-650193c46b28 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:46.577904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d89a63-db85-4eee-a762-9ac6b89e8742 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:46.640028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19045143-ceaa-4a12-a472-fba46e81ed7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:46.640690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19045143-ceaa-4a12-a472-fba46e81ed7c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:46.640969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19045143-ceaa-4a12-a472-fba46e81ed7c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:46.641265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19045143-ceaa-4a12-a472-fba46e81ed7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:46.682014 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b8f60-76c7-4d86-b068-650193c46b28 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:46.730167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:46.747706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb7062-da3d-4b0d-a5f3-21b92823fafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:46.748502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb7062-da3d-4b0d-a5f3-21b92823fafc None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:46.748782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb7062-da3d-4b0d-a5f3-21b92823fafc None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:46.749054 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb7062-da3d-4b0d-a5f3-21b92823fafc None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:46.751050 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb7062-da3d-4b0d-a5f3-21b92823fafc None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:46.753549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb7062-da3d-4b0d-a5f3-21b92823fafc None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:46.762304 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 148/290] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:46 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:46.763444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb7062-da3d-4b0d-a5f3-21b92823fafc None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:46.780337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19045143-ceaa-4a12-a472-fba46e81ed7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:46.781948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19045143-ceaa-4a12-a472-fba46e81ed7c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:46.810624 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:46.949496 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb7062-da3d-4b0d-a5f3-21b92823fafc None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:46.969657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 143/291] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 15:16:46.975569 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19045143-ceaa-4a12-a472-fba46e81ed7c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:47.045958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf7f8a-c980-4c36-a390-11baf3605f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:47.046807 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf7f8a-c980-4c36-a390-11baf3605f4f None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:47.047091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf7f8a-c980-4c36-a390-11baf3605f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:47.047367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf7f8a-c980-4c36-a390-11baf3605f4f None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac/users/44610ee1059c4d9789ba7e3d9f8542e7/roles/64202be313d04a5aab01b866bc558152` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:47.057930 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf7f8a-c980-4c36-a390-11baf3605f4f None admin] RBAC: Authorizing `identity:create_grant(project_id=2bda471206fe4fbb88370df9ab3bdfac, user_id=44610ee1059c4d9789ba7e3d9f8542e7, role_id=64202be313d04a5aab01b866bc558152)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:47.059392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf7f8a-c980-4c36-a390-11baf3605f4f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:47.080614 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:47.101684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19045143-ceaa-4a12-a472-fba46e81ed7c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:47.104187 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 149/292] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:16:46 2022] PUT /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac/users/44610ee1059c4d9789ba7e3d9f8542e7/roles/64202be313d04a5aab01b866bc558152 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:16:47.158983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c4dca-0597-4089-b162-3ce1889c163d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:47.159850 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c4dca-0597-4089-b162-3ce1889c163d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:47.160127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c4dca-0597-4089-b162-3ce1889c163d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:47.160400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c4dca-0597-4089-b162-3ce1889c163d None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:47.161617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c4dca-0597-4089-b162-3ce1889c163d None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:47.163686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c4dca-0597-4089-b162-3ce1889c163d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:47.169926 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 144/293] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:46 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:47.176704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf7f8a-c980-4c36-a390-11baf3605f4f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:47.335872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:47.357119 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf7f8a-c980-4c36-a390-11baf3605f4f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:47.397535 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba2221-b4d4-42e3-b923-7d2df446fd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:47.398200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba2221-b4d4-42e3-b923-7d2df446fd20 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:47.398480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba2221-b4d4-42e3-b923-7d2df446fd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:47.398751 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba2221-b4d4-42e3-b923-7d2df446fd20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:47.399599 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba2221-b4d4-42e3-b923-7d2df446fd20 None admin] RBAC: Adding query filter params (role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f, scope.project.id=2bda471206fe4fbb88370df9ab3bdfac, user.id=e12f0181026843a5aa4fad5423b6ec2a) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:47.399909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba2221-b4d4-42e3-b923-7d2df446fd20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:47.401395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba2221-b4d4-42e3-b923-7d2df446fd20 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:47.408036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 150/294] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:47 2022] GET /identity/v3/role_assignments?user.id=e12f0181026843a5aa4fad5423b6ec2a&scope.project.id=2bda471206fe4fbb88370df9ab3bdfac&role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:47.739029 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49696434-42e5-4307-981f-0fd34a688b9b None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:47.739732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49696434-42e5-4307-981f-0fd34a688b9b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:47.740766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49696434-42e5-4307-981f-0fd34a688b9b None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:47.742388 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 145/295] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:47.747865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7771dbd-fea9-4d82-8f1a-18a0f6f98172 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:47.748162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7771dbd-fea9-4d82-8f1a-18a0f6f98172 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:47.748444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7771dbd-fea9-4d82-8f1a-18a0f6f98172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:47.770535 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-a7771dbd-fea9-4d82-8f1a-18a0f6f98172 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:47.772235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a7771dbd-fea9-4d82-8f1a-18a0f6f98172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:47.843202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 151/296] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:47.868635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49696434-42e5-4307-981f-0fd34a688b9b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:47.999571 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49696434-42e5-4307-981f-0fd34a688b9b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:48.041947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d88ad9-e371-4e7a-938a-242f0ad65fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:48.042989 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d88ad9-e371-4e7a-938a-242f0ad65fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:48.043440 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d88ad9-e371-4e7a-938a-242f0ad65fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:48.043862 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d88ad9-e371-4e7a-938a-242f0ad65fc0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:48.054841 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d88ad9-e371-4e7a-938a-242f0ad65fc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-94d88ad9-e371-4e7a-938a-242f0ad65fc0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:48.062829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:48.063907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:48.064886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:48.064886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:48.064886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:48.064886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:48.064886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:48.064886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:48.064886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:48.064886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:48.064886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:48.064886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:48.064886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:48.064886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:48.067043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 146/297] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:48.074246 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7771dbd-fea9-4d82-8f1a-18a0f6f98172 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:48.153758 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:48.183529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7771dbd-fea9-4d82-8f1a-18a0f6f98172 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:48.219388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c79d7-9978-4e60-8e98-479c859447df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:48.220310 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c79d7-9978-4e60-8e98-479c859447df None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:48.220743 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c79d7-9978-4e60-8e98-479c859447df None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:48.221218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c79d7-9978-4e60-8e98-479c859447df None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:48.222121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c79d7-9978-4e60-8e98-479c859447df None admin] RBAC: Adding query filter params (name=member) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:48.222574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c79d7-9978-4e60-8e98-479c859447df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:48.224093 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c79d7-9978-4e60-8e98-479c859447df None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:48.234393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 152/298] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:48.240117 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d88ad9-e371-4e7a-938a-242f0ad65fc0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:48.257500 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:48.289222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1a5e2-892f-45a9-8cc7-d5d34fc0f4d2 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:48.289515 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1a5e2-892f-45a9-8cc7-d5d34fc0f4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:48.289791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1a5e2-892f-45a9-8cc7-d5d34fc0f4d2 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:48.291446 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 153/299] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:48.296112 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21599b0-b55c-4800-b643-135cd8c14ec1 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:48.296671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21599b0-b55c-4800-b643-135cd8c14ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:48.297064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21599b0-b55c-4800-b643-135cd8c14ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:48.317899 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-e21599b0-b55c-4800-b643-135cd8c14ec1 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:48.319872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e21599b0-b55c-4800-b643-135cd8c14ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:48.354945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d88ad9-e371-4e7a-938a-242f0ad65fc0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:48.389700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755aaf27-10d5-41cb-a82a-80f7207acc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:48.390574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755aaf27-10d5-41cb-a82a-80f7207acc3e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:48.390857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755aaf27-10d5-41cb-a82a-80f7207acc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:48.391132 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755aaf27-10d5-41cb-a82a-80f7207acc3e None admin] PATH_INFO: `/v3/users/01bafcfad5c44384b68490bb40abc8b5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:48.392336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755aaf27-10d5-41cb-a82a-80f7207acc3e None admin] RBAC: Authorizing `identity:get_user(user_id=01bafcfad5c44384b68490bb40abc8b5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:48.393844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755aaf27-10d5-41cb-a82a-80f7207acc3e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:48.400651 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 147/300] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:48 2022] GET /identity/v3/users/01bafcfad5c44384b68490bb40abc8b5 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:48.401866 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755aaf27-10d5-41cb-a82a-80f7207acc3e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:48.410397 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 154/301] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:48.433521 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21599b0-b55c-4800-b643-135cd8c14ec1 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:48.490338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:48.499048 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:48.519862 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755aaf27-10d5-41cb-a82a-80f7207acc3e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:48.535457 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21599b0-b55c-4800-b643-135cd8c14ec1 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:48.560743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a58374-a799-4032-91e2-ff6b42665cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:48.562024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a58374-a799-4032-91e2-ff6b42665cba None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:48.562550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a58374-a799-4032-91e2-ff6b42665cba None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:48.563183 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a58374-a799-4032-91e2-ff6b42665cba None admin] PATH_INFO: `/v3/system/users/01bafcfad5c44384b68490bb40abc8b5/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:48.565173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a58374-a799-4032-91e2-ff6b42665cba None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=01bafcfad5c44384b68490bb40abc8b5, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:48.567106 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a58374-a799-4032-91e2-ff6b42665cba None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:48.572066 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd4b57-4154-4d0c-8031-05cac65bc7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:48.573250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd4b57-4154-4d0c-8031-05cac65bc7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:48.573749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd4b57-4154-4d0c-8031-05cac65bc7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:48.574039 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd4b57-4154-4d0c-8031-05cac65bc7e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:48.585322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 148/302] 10.222.0.42 () {58 vars in 1334 bytes} [Tue Jul 19 15:16:48 2022] PUT /identity/v3/system/users/01bafcfad5c44384b68490bb40abc8b5/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:16:48.597130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd4b57-4154-4d0c-8031-05cac65bc7e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-62bd4b57-4154-4d0c-8031-05cac65bc7e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:48.605646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:48.606647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:48.607454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:48.607454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:48.607454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:48.607454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:48.607454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:48.607454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:48.607454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:48.607454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:48.607454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:48.607454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:48.607454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:48.607454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:48.607454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 155/303] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:16:48.612897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a58374-a799-4032-91e2-ff6b42665cba None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:48.629971 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09d48b-df7f-4003-8908-dc9dd89d28f2 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:48.630272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09d48b-df7f-4003-8908-dc9dd89d28f2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:48.630534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09d48b-df7f-4003-8908-dc9dd89d28f2 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:48.633749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 156/304] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:16:48.639321 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf464f29-dbe4-45dd-8297-edbe3f41002c None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:48.639632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf464f29-dbe4-45dd-8297-edbe3f41002c None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:48.639875 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf464f29-dbe4-45dd-8297-edbe3f41002c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:48.663754 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-cf464f29-dbe4-45dd-8297-edbe3f41002c None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:48.665010 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cf464f29-dbe4-45dd-8297-edbe3f41002c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:48.746714 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 157/305] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:48.755958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a58374-a799-4032-91e2-ff6b42665cba None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:48.768773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf464f29-dbe4-45dd-8297-edbe3f41002c None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:48.795785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524057ac-c893-4ee8-ad5f-6a6178c79c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:48.796463 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524057ac-c893-4ee8-ad5f-6a6178c79c9c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:48.796654 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524057ac-c893-4ee8-ad5f-6a6178c79c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:48.796931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524057ac-c893-4ee8-ad5f-6a6178c79c9c None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:48.797747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524057ac-c893-4ee8-ad5f-6a6178c79c9c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:48.798060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524057ac-c893-4ee8-ad5f-6a6178c79c9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:48.799434 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524057ac-c893-4ee8-ad5f-6a6178c79c9c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:48.809176 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 149/306] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:48.814503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524057ac-c893-4ee8-ad5f-6a6178c79c9c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:48.817162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:48.845389 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:48.895284 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf464f29-dbe4-45dd-8297-edbe3f41002c None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:48.934211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c309e-ad1f-4128-b7ac-627bd8fbb40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:48.935344 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c309e-ad1f-4128-b7ac-627bd8fbb40b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:48.935629 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c309e-ad1f-4128-b7ac-627bd8fbb40b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:48.935948 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c309e-ad1f-4128-b7ac-627bd8fbb40b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:48.937290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c309e-ad1f-4128-b7ac-627bd8fbb40b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:48.938743 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c309e-ad1f-4128-b7ac-627bd8fbb40b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:48.940532 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 158/307] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:48.942738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524057ac-c893-4ee8-ad5f-6a6178c79c9c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:48.946013 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c309e-ad1f-4128-b7ac-627bd8fbb40b None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:48.988651 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb386f6-724d-4397-9801-57145c3ae10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:48.989936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb386f6-724d-4397-9801-57145c3ae10d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:48.990724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb386f6-724d-4397-9801-57145c3ae10d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:48.991896 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb386f6-724d-4397-9801-57145c3ae10d None admin] PATH_INFO: `/v3/users/a5ab5da9dd2a485da86428c8ef8f85af` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:49.008217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb386f6-724d-4397-9801-57145c3ae10d None admin] RBAC: Authorizing `identity:get_user(user_id=a5ab5da9dd2a485da86428c8ef8f85af)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:49.009722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb386f6-724d-4397-9801-57145c3ae10d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:49.016139 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 150/308] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:48 2022] GET /identity/v3/users/a5ab5da9dd2a485da86428c8ef8f85af => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:49.017270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb386f6-724d-4397-9801-57145c3ae10d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:49.074958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c309e-ad1f-4128-b7ac-627bd8fbb40b None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:49.098257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:49.100587 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:49.114083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f3b1e-e7af-4b36-a3a5-8f699dab969f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:49.115974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f3b1e-e7af-4b36-a3a5-8f699dab969f None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:49.115974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f3b1e-e7af-4b36-a3a5-8f699dab969f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:49.116347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f3b1e-e7af-4b36-a3a5-8f699dab969f None admin] PATH_INFO: `/v3/groups` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:49.117161 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f3b1e-e7af-4b36-a3a5-8f699dab969f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:49.118751 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f3b1e-e7af-4b36-a3a5-8f699dab969f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:49.136355 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 159/309] 10.222.0.42 () {60 vars in 1134 bytes} [Tue Jul 19 15:16:48 2022] POST /identity/v3/groups => generated 229 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:16:49.158679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb386f6-724d-4397-9801-57145c3ae10d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:49.200999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6a200-00be-4f83-aef0-f32558e84fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:49.202039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6a200-00be-4f83-aef0-f32558e84fcf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:49.202498 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6a200-00be-4f83-aef0-f32558e84fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:49.202974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6a200-00be-4f83-aef0-f32558e84fcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:49.204129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6a200-00be-4f83-aef0-f32558e84fcf None admin] RBAC: Adding query filter params (role.id=64202be313d04a5aab01b866bc558152, user.id=a5ab5da9dd2a485da86428c8ef8f85af, scope.system=all) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:49.204629 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6a200-00be-4f83-aef0-f32558e84fcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:49.206310 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6a200-00be-4f83-aef0-f32558e84fcf None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:49.213693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 151/310] 10.222.0.42 () {56 vars in 1308 bytes} [Tue Jul 19 15:16:49 2022] GET /identity/v3/role_assignments?user.id=a5ab5da9dd2a485da86428c8ef8f85af&scope.system=all&role.id=64202be313d04a5aab01b866bc558152 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:49.502376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c30d12-5851-4dfa-9af3-e685498bc4a7 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:49.502826 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c30d12-5851-4dfa-9af3-e685498bc4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:49.503196 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c30d12-5851-4dfa-9af3-e685498bc4a7 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:49.504656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 160/311] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:49.510616 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028c8a0-b90b-4767-950f-ac8ac59ba18c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:49.510991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028c8a0-b90b-4767-950f-ac8ac59ba18c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:49.511339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028c8a0-b90b-4767-950f-ac8ac59ba18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:49.529711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8028c8a0-b90b-4767-950f-ac8ac59ba18c None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:49.530910 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8028c8a0-b90b-4767-950f-ac8ac59ba18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:49.603265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 152/312] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:49.625877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c30d12-5851-4dfa-9af3-e685498bc4a7 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:49.739029 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c30d12-5851-4dfa-9af3-e685498bc4a7 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:49.781028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4270eb2-51d5-424d-8e76-e69ebc1cdeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:49.781762 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4270eb2-51d5-424d-8e76-e69ebc1cdeff None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:49.782049 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4270eb2-51d5-424d-8e76-e69ebc1cdeff None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:49.782326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4270eb2-51d5-424d-8e76-e69ebc1cdeff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:49.794498 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4270eb2-51d5-424d-8e76-e69ebc1cdeff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-e4270eb2-51d5-424d-8e76-e69ebc1cdeff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:49.803850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:49.804810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:49.805641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:49.805641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:49.805641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:49.805641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:49.805641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:49.805641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:49.805641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:49.805641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:49.805641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:49.805641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:49.805641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:49.805641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:49.805641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 161/313] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:49.811239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028c8a0-b90b-4767-950f-ac8ac59ba18c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:49.873965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:49.917970 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028c8a0-b90b-4767-950f-ac8ac59ba18c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:49.953932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89796c6b-dcb3-40dc-bed8-ad457c723d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:49.954605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89796c6b-dcb3-40dc-bed8-ad457c723d2a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:49.954884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89796c6b-dcb3-40dc-bed8-ad457c723d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:49.955191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89796c6b-dcb3-40dc-bed8-ad457c723d2a None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:49.955906 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89796c6b-dcb3-40dc-bed8-ad457c723d2a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:49.956209 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89796c6b-dcb3-40dc-bed8-ad457c723d2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:49.957604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89796c6b-dcb3-40dc-bed8-ad457c723d2a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:49.966230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 153/314] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:49.971776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4270eb2-51d5-424d-8e76-e69ebc1cdeff None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:50.020513 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:50.077985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4270eb2-51d5-424d-8e76-e69ebc1cdeff None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:50.111954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fdb89-2a79-4bef-8af4-60361b309608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:50.112887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fdb89-2a79-4bef-8af4-60361b309608 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:50.113197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fdb89-2a79-4bef-8af4-60361b309608 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:50.113476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fdb89-2a79-4bef-8af4-60361b309608 None admin] PATH_INFO: `/v3/users/44610ee1059c4d9789ba7e3d9f8542e7` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:50.114668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fdb89-2a79-4bef-8af4-60361b309608 None admin] RBAC: Authorizing `identity:get_user(user_id=44610ee1059c4d9789ba7e3d9f8542e7)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:50.116876 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fdb89-2a79-4bef-8af4-60361b309608 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:50.118833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 162/315] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:49 2022] GET /identity/v3/users/44610ee1059c4d9789ba7e3d9f8542e7 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:50.124298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89796c6b-dcb3-40dc-bed8-ad457c723d2a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:50.227192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89796c6b-dcb3-40dc-bed8-ad457c723d2a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:50.265537 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:50.273883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a19a48-b4ee-425d-b0de-cd1cd01a5cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:50.274557 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a19a48-b4ee-425d-b0de-cd1cd01a5cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:50.275275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a19a48-b4ee-425d-b0de-cd1cd01a5cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:50.275554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a19a48-b4ee-425d-b0de-cd1cd01a5cf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:50.348437 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a19a48-b4ee-425d-b0de-cd1cd01a5cf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:50.350983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a19a48-b4ee-425d-b0de-cd1cd01a5cf6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:50.456941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 154/316] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:50.462724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fdb89-2a79-4bef-8af4-60361b309608 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:50.566494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fdb89-2a79-4bef-8af4-60361b309608 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:50.579791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:50.606464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf28bb7-551e-4a61-8750-0918860a4fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:50.607263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf28bb7-551e-4a61-8750-0918860a4fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:50.607553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf28bb7-551e-4a61-8750-0918860a4fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:50.607833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf28bb7-551e-4a61-8750-0918860a4fc1 None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:50.609033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf28bb7-551e-4a61-8750-0918860a4fc1 None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:50.610518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf28bb7-551e-4a61-8750-0918860a4fc1 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:50.612550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 163/317] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:50 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:50.618880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a19a48-b4ee-425d-b0de-cd1cd01a5cf6 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:50.641996 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:50.722614 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a19a48-b4ee-425d-b0de-cd1cd01a5cf6 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:50.758171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378d479-22f5-4003-bca7-4adc1a38a43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:50.758839 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378d479-22f5-4003-bca7-4adc1a38a43f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:50.759122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378d479-22f5-4003-bca7-4adc1a38a43f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:50.759399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378d479-22f5-4003-bca7-4adc1a38a43f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:50.760245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378d479-22f5-4003-bca7-4adc1a38a43f None admin] RBAC: Adding query filter params (role.id=64202be313d04a5aab01b866bc558152, scope.project.id=2bda471206fe4fbb88370df9ab3bdfac, user.id=44610ee1059c4d9789ba7e3d9f8542e7) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:50.760558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378d479-22f5-4003-bca7-4adc1a38a43f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:50.762038 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378d479-22f5-4003-bca7-4adc1a38a43f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:50.768502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 155/318] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:16:50 2022] GET /identity/v3/role_assignments?user.id=44610ee1059c4d9789ba7e3d9f8542e7&scope.project.id=2bda471206fe4fbb88370df9ab3bdfac&role.id=64202be313d04a5aab01b866bc558152 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:50.816400 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ede6c-d06d-4f34-86cd-a9ee42b44b38 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:50.816726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ede6c-d06d-4f34-86cd-a9ee42b44b38 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:50.816964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ede6c-d06d-4f34-86cd-a9ee42b44b38 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:50.819368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 164/319] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:50.825320 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9cc83-b1da-4f1b-8248-dd17acee0382 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:50.825880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9cc83-b1da-4f1b-8248-dd17acee0382 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:50.826305 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9cc83-b1da-4f1b-8248-dd17acee0382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:50.848564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6ed9cc83-b1da-4f1b-8248-dd17acee0382 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:50.850371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6ed9cc83-b1da-4f1b-8248-dd17acee0382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:50.927191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 156/320] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:50.950706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ede6c-d06d-4f34-86cd-a9ee42b44b38 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:51.064107 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ede6c-d06d-4f34-86cd-a9ee42b44b38 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:51.099727 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33949394-bc23-4ffb-b4b2-941f56eab985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:51.101523 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33949394-bc23-4ffb-b4b2-941f56eab985 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:51.102064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33949394-bc23-4ffb-b4b2-941f56eab985 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:51.102538 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33949394-bc23-4ffb-b4b2-941f56eab985 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:51.113376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33949394-bc23-4ffb-b4b2-941f56eab985 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-33949394-bc23-4ffb-b4b2-941f56eab985 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:51.121898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:51.122778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:51.123592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:51.123592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:51.123592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:51.123592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:51.123592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:51.123592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:51.123592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:51.123592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:51.123592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:51.123592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:51.123592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:51.123592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:51.125299 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 165/321] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:51.131684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9cc83-b1da-4f1b-8248-dd17acee0382 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:51.144075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:51.217929 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2340a-78bd-41fb-9133-1055db233bce None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:51.218243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2340a-78bd-41fb-9133-1055db233bce None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:51.218395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2340a-78bd-41fb-9133-1055db233bce None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:51.219781 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 166/322] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:16:51.224650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5c6f4-58d0-4076-bad7-1bc779d4d0e6 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:51.224937 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5c6f4-58d0-4076-bad7-1bc779d4d0e6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:51.225243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5c6f4-58d0-4076-bad7-1bc779d4d0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:51.236383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9cc83-b1da-4f1b-8248-dd17acee0382 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:51.245159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-9ee5c6f4-58d0-4076-bad7-1bc779d4d0e6 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:51.246361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9ee5c6f4-58d0-4076-bad7-1bc779d4d0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:51.277706 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a97e4-070b-4e4f-856b-f9fbfd7d0ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:51.278382 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a97e4-070b-4e4f-856b-f9fbfd7d0ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:51.278672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a97e4-070b-4e4f-856b-f9fbfd7d0ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:51.278943 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a97e4-070b-4e4f-856b-f9fbfd7d0ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:51.279697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a97e4-070b-4e4f-856b-f9fbfd7d0ae2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:51.280005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a97e4-070b-4e4f-856b-f9fbfd7d0ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:51.281509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a97e4-070b-4e4f-856b-f9fbfd7d0ae2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:51.284580 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:51.290247 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 157/323] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:51.295504 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a97e4-070b-4e4f-856b-f9fbfd7d0ae2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:51.320928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 167/324] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:51.343278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5c6f4-58d0-4076-bad7-1bc779d4d0e6 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:51.369775 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:51.400764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a97e4-070b-4e4f-856b-f9fbfd7d0ae2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:51.434106 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de57e08-a193-4f99-b268-596bd1c4be15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:51.434987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de57e08-a193-4f99-b268-596bd1c4be15 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:51.435270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de57e08-a193-4f99-b268-596bd1c4be15 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:51.435554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de57e08-a193-4f99-b268-596bd1c4be15 None admin] PATH_INFO: `/v3/users/01bafcfad5c44384b68490bb40abc8b5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:51.436771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de57e08-a193-4f99-b268-596bd1c4be15 None admin] RBAC: Authorizing `identity:get_user(user_id=01bafcfad5c44384b68490bb40abc8b5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:51.438419 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de57e08-a193-4f99-b268-596bd1c4be15 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:51.440058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 158/325] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:51 2022] GET /identity/v3/users/01bafcfad5c44384b68490bb40abc8b5 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:51.442883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14508405-b00c-4169-b66f-482af8698306 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:51.443208 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14508405-b00c-4169-b66f-482af8698306 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:51.443748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14508405-b00c-4169-b66f-482af8698306 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:51.445265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 159/326] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:51.445968 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14508405-b00c-4169-b66f-482af8698306 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:51.446512 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5c6f4-58d0-4076-bad7-1bc779d4d0e6 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:51.483107 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7444ec-53d2-44b4-a1ab-f2eb52988965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:51.483903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7444ec-53d2-44b4-a1ab-f2eb52988965 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:51.484191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7444ec-53d2-44b4-a1ab-f2eb52988965 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:51.484471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7444ec-53d2-44b4-a1ab-f2eb52988965 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:51.485656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7444ec-53d2-44b4-a1ab-f2eb52988965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:51.487110 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7444ec-53d2-44b4-a1ab-f2eb52988965 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:51.488740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 168/327] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:16:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:16:51.490054 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d72e05-3c25-4b4a-9743-fe8365511a9c None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:51.490249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d72e05-3c25-4b4a-9743-fe8365511a9c None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:51.490528 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d72e05-3c25-4b4a-9743-fe8365511a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:51.506517 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-f0d72e05-3c25-4b4a-9743-fe8365511a9c None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:51.507716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f0d72e05-3c25-4b4a-9743-fe8365511a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:51.538901 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:51.553417 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14508405-b00c-4169-b66f-482af8698306 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:51.576800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 169/328] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:51.582023 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d72e05-3c25-4b4a-9743-fe8365511a9c None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:51.587323 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9bfe3-a576-46f8-9270-89152d32546e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:51.587979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9bfe3-a576-46f8-9270-89152d32546e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:51.588261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9bfe3-a576-46f8-9270-89152d32546e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:51.588539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9bfe3-a576-46f8-9270-89152d32546e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:51.589404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9bfe3-a576-46f8-9270-89152d32546e None admin] RBAC: Adding query filter params (role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f, user.id=01bafcfad5c44384b68490bb40abc8b5, scope.system=all) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:51.589743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9bfe3-a576-46f8-9270-89152d32546e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:51.591141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9bfe3-a576-46f8-9270-89152d32546e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:51.596874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:51.614432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 160/329] 10.222.0.42 () {56 vars in 1308 bytes} [Tue Jul 19 15:16:51 2022] GET /identity/v3/role_assignments?user.id=01bafcfad5c44384b68490bb40abc8b5&scope.system=all&role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 510 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:51.614432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9bfe3-a576-46f8-9270-89152d32546e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:51.696786 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d72e05-3c25-4b4a-9743-fe8365511a9c None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:51.730460 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9bfe3-a576-46f8-9270-89152d32546e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:51.734894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e86f4e-80a4-499a-83ea-1940f5dd262e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:51.735511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e86f4e-80a4-499a-83ea-1940f5dd262e None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:51.735793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e86f4e-80a4-499a-83ea-1940f5dd262e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:51.736070 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e86f4e-80a4-499a-83ea-1940f5dd262e None admin] PATH_INFO: `/v3/groups` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:51.736771 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e86f4e-80a4-499a-83ea-1940f5dd262e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:51.738284 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e86f4e-80a4-499a-83ea-1940f5dd262e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:51.755007 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 170/330] 10.222.0.42 () {60 vars in 1134 bytes} [Tue Jul 19 15:16:51 2022] POST /identity/v3/groups => generated 226 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:16:51.770403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4717575-0ba8-4663-ad61-a00800b83f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:51.771795 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4717575-0ba8-4663-ad61-a00800b83f48 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:51.772082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4717575-0ba8-4663-ad61-a00800b83f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:51.772353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4717575-0ba8-4663-ad61-a00800b83f48 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:51.784932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4717575-0ba8-4663-ad61-a00800b83f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e4717575-0ba8-4663-ad61-a00800b83f48 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:51.793129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:51.794299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:51.795130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:51.795130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:51.795130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:51.795130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:51.795130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:51.795130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:51.795130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:51.795130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:51.795130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:51.795130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:51.795130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:51.795130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:51.795130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 161/331] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:51.800940 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e86f4e-80a4-499a-83ea-1940f5dd262e None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:51.835217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:51.915120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e86f4e-80a4-499a-83ea-1940f5dd262e None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:51.952110 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0a633-db46-44fd-9b2a-c52fb26783c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:51.952780 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0a633-db46-44fd-9b2a-c52fb26783c7 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:51.953061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0a633-db46-44fd-9b2a-c52fb26783c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:51.953420 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0a633-db46-44fd-9b2a-c52fb26783c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:51.954168 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0a633-db46-44fd-9b2a-c52fb26783c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:51.954473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0a633-db46-44fd-9b2a-c52fb26783c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:51.955850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0a633-db46-44fd-9b2a-c52fb26783c7 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:51.964093 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 171/332] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:51.969957 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4717575-0ba8-4663-ad61-a00800b83f48 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:52.033799 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:52.082523 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4717575-0ba8-4663-ad61-a00800b83f48 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:52.118960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc9bde-117a-4ed7-91b8-f0f0f916066a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:52.119869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc9bde-117a-4ed7-91b8-f0f0f916066a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:52.120141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc9bde-117a-4ed7-91b8-f0f0f916066a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:52.120415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc9bde-117a-4ed7-91b8-f0f0f916066a None admin] PATH_INFO: `/v3/users/a5ab5da9dd2a485da86428c8ef8f85af` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:52.121726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc9bde-117a-4ed7-91b8-f0f0f916066a None admin] RBAC: Authorizing `identity:get_user(user_id=a5ab5da9dd2a485da86428c8ef8f85af)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:52.123174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc9bde-117a-4ed7-91b8-f0f0f916066a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:52.124996 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 162/333] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:51 2022] GET /identity/v3/users/a5ab5da9dd2a485da86428c8ef8f85af => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:52.131183 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0a633-db46-44fd-9b2a-c52fb26783c7 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:52.231882 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:52.245047 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0a633-db46-44fd-9b2a-c52fb26783c7 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:52.284433 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811841c-8bee-43e5-a041-0728f5cd6f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:52.285234 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811841c-8bee-43e5-a041-0728f5cd6f72 None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:52.285518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811841c-8bee-43e5-a041-0728f5cd6f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:52.285795 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811841c-8bee-43e5-a041-0728f5cd6f72 None admin] PATH_INFO: `/v3/system/users/a5ab5da9dd2a485da86428c8ef8f85af/roles/64202be313d04a5aab01b866bc558152` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:52.287829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811841c-8bee-43e5-a041-0728f5cd6f72 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a5ab5da9dd2a485da86428c8ef8f85af, role_id=64202be313d04a5aab01b866bc558152)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:52.289263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811841c-8bee-43e5-a041-0728f5cd6f72 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:52.303033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 172/334] 10.222.0.42 () {58 vars in 1334 bytes} [Tue Jul 19 15:16:52 2022] PUT /identity/v3/system/users/a5ab5da9dd2a485da86428c8ef8f85af/roles/64202be313d04a5aab01b866bc558152 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:16:54.605059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db540eb-a7b8-4640-81f5-f699c0086308 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:54.605585 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db540eb-a7b8-4640-81f5-f699c0086308 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:54.605585 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db540eb-a7b8-4640-81f5-f699c0086308 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:54.607029 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 163/335] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:54.613736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabf6cd-047b-4678-82d0-b394afde2c12 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:54.614581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabf6cd-047b-4678-82d0-b394afde2c12 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:54.615194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabf6cd-047b-4678-82d0-b394afde2c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:54.652829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-5cabf6cd-047b-4678-82d0-b394afde2c12 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:54.654414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5cabf6cd-047b-4678-82d0-b394afde2c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:54.806917 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 173/336] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:54.848091 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db540eb-a7b8-4640-81f5-f699c0086308 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:54.909122 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccdf5b-6d2f-4aa0-990f-e77546cc7a43 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:54.909710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccdf5b-6d2f-4aa0-990f-e77546cc7a43 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:54.910167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccdf5b-6d2f-4aa0-990f-e77546cc7a43 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:54.911773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 174/337] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:16:54.919213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5624cb6-d84d-41c9-9868-5d34eae640ec None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:54.919733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5624cb6-d84d-41c9-9868-5d34eae640ec None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:54.920210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5624cb6-d84d-41c9-9868-5d34eae640ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:54.955001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-a5624cb6-d84d-41c9-9868-5d34eae640ec None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:54.958163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a5624cb6-d84d-41c9-9868-5d34eae640ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:55.078447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db540eb-a7b8-4640-81f5-f699c0086308 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:55.133875 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:55.145175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c212e-d26a-4c40-8174-81d0402201ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:55.145175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c212e-d26a-4c40-8174-81d0402201ec None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:55.145175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c212e-d26a-4c40-8174-81d0402201ec None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:55.145175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c212e-d26a-4c40-8174-81d0402201ec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:55.164324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 175/338] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:16:55.186100 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c212e-d26a-4c40-8174-81d0402201ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-009c212e-d26a-4c40-8174-81d0402201ec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:55.194649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:55.195709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:55.196502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:55.196502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:55.196502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:55.196502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:55.196502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:55.196502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:55.196502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:55.196502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:55.196502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:55.196502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:55.196502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 15:16:55.196502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:55.202916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 164/339] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:55.208784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5624cb6-d84d-41c9-9868-5d34eae640ec None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:55.226259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c212e-d26a-4c40-8174-81d0402201ec None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:55.237769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:55.363233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5624cb6-d84d-41c9-9868-5d34eae640ec None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:55.367580 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c212e-d26a-4c40-8174-81d0402201ec None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:55.404905 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e626738-b41d-45b4-a510-e7e5152c7eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:55.405126 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e626738-b41d-45b4-a510-e7e5152c7eba None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:55.405436 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e626738-b41d-45b4-a510-e7e5152c7eba None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:55.405712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e626738-b41d-45b4-a510-e7e5152c7eba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:55.410746 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb35d92-f8da-4f00-94b6-bef3b0b86f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:55.412354 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb35d92-f8da-4f00-94b6-bef3b0b86f19 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:55.412832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb35d92-f8da-4f00-94b6-bef3b0b86f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:55.413436 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb35d92-f8da-4f00-94b6-bef3b0b86f19 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:55.414709 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb35d92-f8da-4f00-94b6-bef3b0b86f19 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:55.415422 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb35d92-f8da-4f00-94b6-bef3b0b86f19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:55.417429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb35d92-f8da-4f00-94b6-bef3b0b86f19 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:55.418722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e626738-b41d-45b4-a510-e7e5152c7eba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:55.427753 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 176/340] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:55.429388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c12a35-8cf4-4eaf-8c24-a23f98c5a488 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:55.429874 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c12a35-8cf4-4eaf-8c24-a23f98c5a488 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-4e626738-b41d-45b4-a510-e7e5152c7eba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:55.429977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:55.430804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:55.431595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:55.431595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:55.431595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:55.431595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:55.431595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:55.431595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:55.431595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:55.431595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:55.431595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:55.431595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:55.431595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:55.431595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:55.431595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 165/341] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:16:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:55.432754 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f80cd-0492-49e0-a6e1-300d48f61729 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:55.432932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f80cd-0492-49e0-a6e1-300d48f61729 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:55.433242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f80cd-0492-49e0-a6e1-300d48f61729 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:55.433633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c12a35-8cf4-4eaf-8c24-a23f98c5a488 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:55.434664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 166/342] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:55.435815 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 177/343] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:55.440982 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f80cd-0492-49e0-a6e1-300d48f61729 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:55.441123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c12a35-8cf4-4eaf-8c24-a23f98c5a488 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:55.514655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:55.549461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:55.577831 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c12a35-8cf4-4eaf-8c24-a23f98c5a488 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:55.582722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f80cd-0492-49e0-a6e1-300d48f61729 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:55.616482 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae6f43-d149-486f-b53f-2330802af514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:55.617175 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae6f43-d149-486f-b53f-2330802af514 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:55.617461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae6f43-d149-486f-b53f-2330802af514 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:55.617739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae6f43-d149-486f-b53f-2330802af514 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:55.618500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae6f43-d149-486f-b53f-2330802af514 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:55.618819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae6f43-d149-486f-b53f-2330802af514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:55.620196 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae6f43-d149-486f-b53f-2330802af514 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:55.623638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc910948-3e1a-4fd8-aa26-ac9dbec2e230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:55.623638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc910948-3e1a-4fd8-aa26-ac9dbec2e230 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:55.623638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc910948-3e1a-4fd8-aa26-ac9dbec2e230 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:55.623638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc910948-3e1a-4fd8-aa26-ac9dbec2e230 None admin] PATH_INFO: `/v3/users/a5ab5da9dd2a485da86428c8ef8f85af` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:55.625033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc910948-3e1a-4fd8-aa26-ac9dbec2e230 None admin] RBAC: Authorizing `identity:get_user(user_id=a5ab5da9dd2a485da86428c8ef8f85af)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:55.626698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc910948-3e1a-4fd8-aa26-ac9dbec2e230 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:55.628349 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 167/344] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:16:55 2022] GET /identity/v3/users/a5ab5da9dd2a485da86428c8ef8f85af => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:55.629084 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 178/345] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:16:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:55.629567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89152ae2-542c-4266-97df-783a04b4d51f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:55.629798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89152ae2-542c-4266-97df-783a04b4d51f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:55.630067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89152ae2-542c-4266-97df-783a04b4d51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:55.630780 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784303a-d0ab-42cc-b35f-c7277ab6bae9 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:55.631714 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784303a-d0ab-42cc-b35f-c7277ab6bae9 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:55.632011 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784303a-d0ab-42cc-b35f-c7277ab6bae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:55.649375 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-89152ae2-542c-4266-97df-783a04b4d51f None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:55.650573 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-7784303a-d0ab-42cc-b35f-c7277ab6bae9 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:55.650817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-89152ae2-542c-4266-97df-783a04b4d51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:55.651739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7784303a-d0ab-42cc-b35f-c7277ab6bae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:55.730649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 168/346] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:55.731322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 179/347] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:55.731992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec787d-0170-4b72-b69e-8b6293677772 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:55.732469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec787d-0170-4b72-b69e-8b6293677772 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:55.732749 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec787d-0170-4b72-b69e-8b6293677772 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:55.733511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784303a-d0ab-42cc-b35f-c7277ab6bae9 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:55.734122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 169/348] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:55.734816 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec787d-0170-4b72-b69e-8b6293677772 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:55.793748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:55.826439 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:55.872349 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec787d-0170-4b72-b69e-8b6293677772 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:55.873951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784303a-d0ab-42cc-b35f-c7277ab6bae9 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:55.916725 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90997b6-4ad6-44e3-9e87-414680a300cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:55.917517 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90997b6-4ad6-44e3-9e87-414680a300cd None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:55.917713 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90997b6-4ad6-44e3-9e87-414680a300cd None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:55.917989 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90997b6-4ad6-44e3-9e87-414680a300cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:55.918833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90997b6-4ad6-44e3-9e87-414680a300cd None admin] RBAC: Adding query filter params (role.id=64202be313d04a5aab01b866bc558152, user.id=a5ab5da9dd2a485da86428c8ef8f85af, scope.system=all) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:55.919149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90997b6-4ad6-44e3-9e87-414680a300cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:55.920616 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90997b6-4ad6-44e3-9e87-414680a300cd None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:55.922283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb283af-3f4a-4fe1-9570-b0411a81038f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:55.922939 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb283af-3f4a-4fe1-9570-b0411a81038f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:55.923224 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb283af-3f4a-4fe1-9570-b0411a81038f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:55.923505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb283af-3f4a-4fe1-9570-b0411a81038f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:55.936427 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 180/349] 10.222.0.42 () {56 vars in 1308 bytes} [Tue Jul 19 15:16:55 2022] GET /identity/v3/role_assignments?user.id=a5ab5da9dd2a485da86428c8ef8f85af&scope.system=all&role.id=64202be313d04a5aab01b866bc558152 => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:55.938024 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f1c00-ee20-46aa-88ef-661c7fff0193 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:55.938494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f1c00-ee20-46aa-88ef-661c7fff0193 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:55.938949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f1c00-ee20-46aa-88ef-661c7fff0193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:55.956276 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ac7f1c00-ee20-46aa-88ef-661c7fff0193 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:55.958057 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ac7f1c00-ee20-46aa-88ef-661c7fff0193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:56.013046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb283af-3f4a-4fe1-9570-b0411a81038f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:56.014605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb283af-3f4a-4fe1-9570-b0411a81038f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:56.048618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 181/350] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:56.049949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f1c00-ee20-46aa-88ef-661c7fff0193 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:56.077035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:56.141171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 170/351] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:56.141171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb283af-3f4a-4fe1-9570-b0411a81038f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:56.142782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:56.205626 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f1c00-ee20-46aa-88ef-661c7fff0193 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:56.261847 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c1f20-df74-48a7-8383-344184125e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:56.262669 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c1f20-df74-48a7-8383-344184125e26 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:56.262951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c1f20-df74-48a7-8383-344184125e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:56.263272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c1f20-df74-48a7-8383-344184125e26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:56.280735 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c1f20-df74-48a7-8383-344184125e26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-cf8c1f20-df74-48a7-8383-344184125e26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:56.289653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:56.290560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:56.291364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:56.291364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:56.291364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:56.291364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:56.291364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:56.291364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:56.291364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:56.291364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:56.291364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:56.291364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:56.291364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:56.291364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:56.291966 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb283af-3f4a-4fe1-9570-b0411a81038f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:56.292044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 182/352] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:56.292044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c1f20-df74-48a7-8383-344184125e26 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:56.336030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcfdaa-b089-48f4-b737-8d2e6399d4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:56.337229 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcfdaa-b089-48f4-b737-8d2e6399d4da None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:56.337768 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcfdaa-b089-48f4-b737-8d2e6399d4da None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:56.338045 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcfdaa-b089-48f4-b737-8d2e6399d4da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:56.350211 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcfdaa-b089-48f4-b737-8d2e6399d4da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-57dcfdaa-b089-48f4-b737-8d2e6399d4da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:56.359757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:56.360700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:56.361983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:56.361983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:56.361983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:56.361983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:56.361983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:56.361983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:56.361983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:56.361983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:56.361983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:56.361983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:56.361983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:16:56.361983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:56.361983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 171/353] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:56.361983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcfdaa-b089-48f4-b737-8d2e6399d4da None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:56.366574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:56.416962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c1f20-df74-48a7-8383-344184125e26 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:56.438770 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:56.464649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb7186-ad60-4a54-8f87-93035557823e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:56.465760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb7186-ad60-4a54-8f87-93035557823e None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:56.466467 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb7186-ad60-4a54-8f87-93035557823e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:56.467112 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb7186-ad60-4a54-8f87-93035557823e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:56.479765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb7186-ad60-4a54-8f87-93035557823e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:56.482072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcfdaa-b089-48f4-b737-8d2e6399d4da None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-dabb7186-ad60-4a54-8f87-93035557823e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:56.489648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:56.491148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:56.491956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:56.491956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:56.491956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:56.491956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:56.491956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:56.491956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:56.491956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:56.491956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:56.491956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:56.491956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:56.491956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:16:56.491956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:16:56.491956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 183/354] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:16:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:56.493312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb7186-ad60-4a54-8f87-93035557823e None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:56.524285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc16fe-9b22-48b6-b249-8870a03a0cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:56.525311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc16fe-9b22-48b6-b249-8870a03a0cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:56.525713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc16fe-9b22-48b6-b249-8870a03a0cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:56.526012 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc16fe-9b22-48b6-b249-8870a03a0cd1 None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:56.527539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc16fe-9b22-48b6-b249-8870a03a0cd1 None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:56.530644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc16fe-9b22-48b6-b249-8870a03a0cd1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:56.532589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 172/355] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:56 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:56.533467 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc16fe-9b22-48b6-b249-8870a03a0cd1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:56.611916 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb7186-ad60-4a54-8f87-93035557823e None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:56.614603 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:56.650706 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc16fe-9b22-48b6-b249-8870a03a0cd1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:56.651336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c56cc3-3f5b-45a1-abf1-896cbf33c5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:56.651698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c56cc3-3f5b-45a1-abf1-896cbf33c5ac None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:56.651698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c56cc3-3f5b-45a1-abf1-896cbf33c5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:56.651959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c56cc3-3f5b-45a1-abf1-896cbf33c5ac None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:56.653921 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c56cc3-3f5b-45a1-abf1-896cbf33c5ac None admin] RBAC: Adding query filter params (name=member) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:56.654266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c56cc3-3f5b-45a1-abf1-896cbf33c5ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:56.655676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c56cc3-3f5b-45a1-abf1-896cbf33c5ac None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:56.666954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 184/356] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:56.667930 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59543c7-57bb-4137-ad89-682656d57928 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:56.668227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59543c7-57bb-4137-ad89-682656d57928 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:56.668579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59543c7-57bb-4137-ad89-682656d57928 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:56.669891 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 185/357] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:56.673918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59543c7-57bb-4137-ad89-682656d57928 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:56.691448 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8f033-dfea-4004-99b4-5d979090581f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:56.692135 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8f033-dfea-4004-99b4-5d979090581f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:56.692415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8f033-dfea-4004-99b4-5d979090581f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:56.692745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8f033-dfea-4004-99b4-5d979090581f None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:56.693664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8f033-dfea-4004-99b4-5d979090581f None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:56.693982 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8f033-dfea-4004-99b4-5d979090581f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:56.695367 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8f033-dfea-4004-99b4-5d979090581f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:56.704838 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 173/358] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:16:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:56.705982 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8f033-dfea-4004-99b4-5d979090581f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:56.726030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:56.806062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59543c7-57bb-4137-ad89-682656d57928 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:56.837831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8f033-dfea-4004-99b4-5d979090581f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:56.847083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d504d-86d7-472c-b60e-949cd13afca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:56.847770 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d504d-86d7-472c-b60e-949cd13afca5 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:56.848059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d504d-86d7-472c-b60e-949cd13afca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:56.849455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d504d-86d7-472c-b60e-949cd13afca5 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:56.850739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d504d-86d7-472c-b60e-949cd13afca5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:56.851055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d504d-86d7-472c-b60e-949cd13afca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:56.852453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d504d-86d7-472c-b60e-949cd13afca5 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:56.862423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 186/359] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:16:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:56.863341 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d504d-86d7-472c-b60e-949cd13afca5 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:56.883524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a183fb-85e1-4db4-b47b-4e299fa35ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:56.884401 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a183fb-85e1-4db4-b47b-4e299fa35ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:56.884680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a183fb-85e1-4db4-b47b-4e299fa35ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:56.884954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a183fb-85e1-4db4-b47b-4e299fa35ad8 None admin] PATH_INFO: `/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:56.888953 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:56.891896 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a183fb-85e1-4db4-b47b-4e299fa35ad8 None admin] RBAC: Authorizing `identity:get_group(group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:56.893821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a183fb-85e1-4db4-b47b-4e299fa35ad8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:56.895969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 174/360] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Jul 19 15:16:56 2022] GET /identity/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:56.897149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d04e3e-ff66-41eb-b937-1c96e6733f31 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:56.897462 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d04e3e-ff66-41eb-b937-1c96e6733f31 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:56.897799 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d04e3e-ff66-41eb-b937-1c96e6733f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:56.915739 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e7d04e3e-ff66-41eb-b937-1c96e6733f31 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:56.916938 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e7d04e3e-ff66-41eb-b937-1c96e6733f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:56.988502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:56.990669 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 175/361] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:56.991531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d04e3e-ff66-41eb-b937-1c96e6733f31 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:56.994488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d504d-86d7-472c-b60e-949cd13afca5 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:57.037898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5686894-09c9-473b-9983-88faff194add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:57.037898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5686894-09c9-473b-9983-88faff194add None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:57.037898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5686894-09c9-473b-9983-88faff194add None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:57.038249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5686894-09c9-473b-9983-88faff194add None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:57.107043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d04e3e-ff66-41eb-b937-1c96e6733f31 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:57.123673 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5686894-09c9-473b-9983-88faff194add None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:57.125285 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5686894-09c9-473b-9983-88faff194add None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:57.140450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:57.150427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3474b4a-bdfc-4f4c-8a78-153603e0670c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:57.151094 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3474b4a-bdfc-4f4c-8a78-153603e0670c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:57.151382 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3474b4a-bdfc-4f4c-8a78-153603e0670c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:57.151663 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3474b4a-bdfc-4f4c-8a78-153603e0670c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:57.232441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3474b4a-bdfc-4f4c-8a78-153603e0670c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:57.234015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3474b4a-bdfc-4f4c-8a78-153603e0670c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:57.248245 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:57.254106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 187/362] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:57.255996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5686894-09c9-473b-9983-88faff194add None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:57.369317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 176/363] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:57.369317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3474b4a-bdfc-4f4c-8a78-153603e0670c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:57.384832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5686894-09c9-473b-9983-88faff194add None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:57.398903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:57.429168 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714dfcc-9a4b-441d-820f-ec62f8e986c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:57.429843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714dfcc-9a4b-441d-820f-ec62f8e986c9 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:57.430136 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714dfcc-9a4b-441d-820f-ec62f8e986c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:57.430409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714dfcc-9a4b-441d-820f-ec62f8e986c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:57.483271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3474b4a-bdfc-4f4c-8a78-153603e0670c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:57.510891 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:57.514212 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714dfcc-9a4b-441d-820f-ec62f8e986c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:57.516343 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714dfcc-9a4b-441d-820f-ec62f8e986c9 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:57.526259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6105e5-d5b2-4e5b-a042-707d78ccbea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:57.526923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6105e5-d5b2-4e5b-a042-707d78ccbea2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:57.527469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6105e5-d5b2-4e5b-a042-707d78ccbea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:57.527749 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6105e5-d5b2-4e5b-a042-707d78ccbea2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:57.528609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6105e5-d5b2-4e5b-a042-707d78ccbea2 None admin] RBAC: Adding query filter params (group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445, role.id=44bd561cc5594ffd9bcc27ef6dad0fc5, scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:57.528926 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6105e5-d5b2-4e5b-a042-707d78ccbea2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:57.530397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6105e5-d5b2-4e5b-a042-707d78ccbea2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:57.536061 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 177/364] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Jul 19 15:16:57 2022] GET /identity/v3/role_assignments?group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445&scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5&role.id=44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:57.537531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6105e5-d5b2-4e5b-a042-707d78ccbea2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:57.656989 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:57.661754 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 188/365] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:57.662674 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714dfcc-9a4b-441d-820f-ec62f8e986c9 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:57.669804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6105e5-d5b2-4e5b-a042-707d78ccbea2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:57.709708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf4da0-c614-4774-a29e-76a13603265d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:57.710511 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf4da0-c614-4774-a29e-76a13603265d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:57.710801 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf4da0-c614-4774-a29e-76a13603265d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:57.711082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf4da0-c614-4774-a29e-76a13603265d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:57.721466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf4da0-c614-4774-a29e-76a13603265d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e3cf4da0-c614-4774-a29e-76a13603265d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:16:57.729286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:16:57.730216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:16:57.731065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:16:57.731065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:16:57.731065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:16:57.731065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:16:57.731065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:16:57.731065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:16:57.731065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:16:57.731065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:16:57.731065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:16:57.731065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:16:57.731065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:16:57.731065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:16:57.731065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 178/366] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:16:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:16:57.732367 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf4da0-c614-4774-a29e-76a13603265d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:57.774593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:57.778009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714dfcc-9a4b-441d-820f-ec62f8e986c9 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:57.815136 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abaa7e-0e4e-4855-88cd-9eb3ddda436a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:57.816015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abaa7e-0e4e-4855-88cd-9eb3ddda436a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:57.816387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abaa7e-0e4e-4855-88cd-9eb3ddda436a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:57.816660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abaa7e-0e4e-4855-88cd-9eb3ddda436a None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:57.818000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abaa7e-0e4e-4855-88cd-9eb3ddda436a None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:57.819456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abaa7e-0e4e-4855-88cd-9eb3ddda436a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:57.826036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 189/367] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:57 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:57.826753 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abaa7e-0e4e-4855-88cd-9eb3ddda436a None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:57.839850 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf4da0-c614-4774-a29e-76a13603265d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:57.875730 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdc4b8-197b-4330-b314-fec138fbef05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:57.876588 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdc4b8-197b-4330-b314-fec138fbef05 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:57.876873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdc4b8-197b-4330-b314-fec138fbef05 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:57.877163 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdc4b8-197b-4330-b314-fec138fbef05 None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:57.878558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdc4b8-197b-4330-b314-fec138fbef05 None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:57.879999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdc4b8-197b-4330-b314-fec138fbef05 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:57.882421 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 179/368] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:57 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:57.883198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdc4b8-197b-4330-b314-fec138fbef05 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:57.906961 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:57.938051 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abaa7e-0e4e-4855-88cd-9eb3ddda436a None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:57.972534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24659c-308e-4039-a5b5-44daeb48cd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:57.973431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24659c-308e-4039-a5b5-44daeb48cd47 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:57.973710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24659c-308e-4039-a5b5-44daeb48cd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:57.974555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24659c-308e-4039-a5b5-44daeb48cd47 None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:57.975875 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24659c-308e-4039-a5b5-44daeb48cd47 None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:57.977476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24659c-308e-4039-a5b5-44daeb48cd47 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:57.984603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 190/369] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:57 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:57.984603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24659c-308e-4039-a5b5-44daeb48cd47 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:58.003902 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdc4b8-197b-4330-b314-fec138fbef05 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:58.023794 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:58.043165 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63b58e-60e8-47f6-b35d-ba1ff988dea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:58.043950 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63b58e-60e8-47f6-b35d-ba1ff988dea3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:58.044231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63b58e-60e8-47f6-b35d-ba1ff988dea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:58.044506 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63b58e-60e8-47f6-b35d-ba1ff988dea3 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:58.045364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63b58e-60e8-47f6-b35d-ba1ff988dea3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:58.045673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63b58e-60e8-47f6-b35d-ba1ff988dea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:58.047290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63b58e-60e8-47f6-b35d-ba1ff988dea3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:58.056927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 180/370] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:16:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:58.062092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63b58e-60e8-47f6-b35d-ba1ff988dea3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:58.092067 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24659c-308e-4039-a5b5-44daeb48cd47 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:58.129019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c4e81-953d-41a6-9e11-d2afa3c7e282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:58.129905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c4e81-953d-41a6-9e11-d2afa3c7e282 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:58.130884 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c4e81-953d-41a6-9e11-d2afa3c7e282 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:58.131179 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c4e81-953d-41a6-9e11-d2afa3c7e282 None admin] PATH_INFO: `/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:58.132375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c4e81-953d-41a6-9e11-d2afa3c7e282 None admin] RBAC: Authorizing `identity:get_group(group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:58.133859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c4e81-953d-41a6-9e11-d2afa3c7e282 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:58.135765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 191/371] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Jul 19 15:16:57 2022] GET /identity/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:58.136550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c4e81-953d-41a6-9e11-d2afa3c7e282 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:58.153532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:58.174304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63b58e-60e8-47f6-b35d-ba1ff988dea3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:58.210877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9779e1c-4fe1-4b59-be11-aa1b2717e5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:58.211667 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9779e1c-4fe1-4b59-be11-aa1b2717e5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:58.211955 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9779e1c-4fe1-4b59-be11-aa1b2717e5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:58.212235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9779e1c-4fe1-4b59-be11-aa1b2717e5e3 None admin] PATH_INFO: `/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:58.213446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9779e1c-4fe1-4b59-be11-aa1b2717e5e3 None admin] RBAC: Authorizing `identity:get_group(group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:58.214895 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9779e1c-4fe1-4b59-be11-aa1b2717e5e3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:58.216532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 181/372] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Jul 19 15:16:58 2022] GET /identity/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:58.217338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9779e1c-4fe1-4b59-be11-aa1b2717e5e3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:58.260508 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c4e81-953d-41a6-9e11-d2afa3c7e282 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:58.263574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:58.299733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ebbbc-677f-4fd2-b0f8-906f33e85581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:58.300592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ebbbc-677f-4fd2-b0f8-906f33e85581 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:58.300872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ebbbc-677f-4fd2-b0f8-906f33e85581 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:58.301179 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ebbbc-677f-4fd2-b0f8-906f33e85581 None admin] PATH_INFO: `/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:58.302601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ebbbc-677f-4fd2-b0f8-906f33e85581 None admin] RBAC: Authorizing `identity:get_group(group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:58.304446 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ebbbc-677f-4fd2-b0f8-906f33e85581 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:58.311555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 192/373] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Jul 19 15:16:58 2022] GET /identity/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:58.311555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ebbbc-677f-4fd2-b0f8-906f33e85581 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:58.329365 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9779e1c-4fe1-4b59-be11-aa1b2717e5e3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:58.365901 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020b834-3845-4ed1-8fcb-6e983730c0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:58.366654 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020b834-3845-4ed1-8fcb-6e983730c0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:58.366943 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020b834-3845-4ed1-8fcb-6e983730c0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:58.367220 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020b834-3845-4ed1-8fcb-6e983730c0c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:58.386847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:58.415554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ebbbc-677f-4fd2-b0f8-906f33e85581 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:58.439250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020b834-3845-4ed1-8fcb-6e983730c0c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:58.440700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020b834-3845-4ed1-8fcb-6e983730c0c8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:58.451095 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191c33d-f651-42ef-aee3-df2196391da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:58.452017 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191c33d-f651-42ef-aee3-df2196391da6 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:58.452454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191c33d-f651-42ef-aee3-df2196391da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:58.452892 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191c33d-f651-42ef-aee3-df2196391da6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:58.453895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191c33d-f651-42ef-aee3-df2196391da6 None admin] RBAC: Adding query filter params (group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445, role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f, scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:58.454379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191c33d-f651-42ef-aee3-df2196391da6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:58.455928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191c33d-f651-42ef-aee3-df2196391da6 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:58.473268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 193/374] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Jul 19 15:16:58 2022] GET /identity/v3/role_assignments?group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445&scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5&role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 260 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:58.473268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191c33d-f651-42ef-aee3-df2196391da6 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:58.514326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:58.585618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 182/375] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:16:58.586604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020b834-3845-4ed1-8fcb-6e983730c0c8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:58.590509 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191c33d-f651-42ef-aee3-df2196391da6 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:58.632048 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebb573-f547-41ec-bd24-d8910d4096f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:58.632715 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebb573-f547-41ec-bd24-d8910d4096f1 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:58.633001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebb573-f547-41ec-bd24-d8910d4096f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:58.633314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebb573-f547-41ec-bd24-d8910d4096f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:58.634163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebb573-f547-41ec-bd24-d8910d4096f1 None admin] RBAC: Adding query filter params (group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445, role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f, scope.project.id=2bda471206fe4fbb88370df9ab3bdfac) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:58.634484 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebb573-f547-41ec-bd24-d8910d4096f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:58.635942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebb573-f547-41ec-bd24-d8910d4096f1 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:58.642864 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 194/376] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Jul 19 15:16:58 2022] GET /identity/v3/role_assignments?group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445&scope.project.id=2bda471206fe4fbb88370df9ab3bdfac&role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:58.643691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebb573-f547-41ec-bd24-d8910d4096f1 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:58.656650 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:58.719039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020b834-3845-4ed1-8fcb-6e983730c0c8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:58.760780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173c888-7cab-489c-aaa5-b755e20302a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:58.761502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173c888-7cab-489c-aaa5-b755e20302a3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:58.761784 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173c888-7cab-489c-aaa5-b755e20302a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:58.762060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173c888-7cab-489c-aaa5-b755e20302a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:58.763022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173c888-7cab-489c-aaa5-b755e20302a3 None admin] RBAC: Adding query filter params (group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445, role.id=44bd561cc5594ffd9bcc27ef6dad0fc5, scope.project.id=2bda471206fe4fbb88370df9ab3bdfac) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:58.763341 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173c888-7cab-489c-aaa5-b755e20302a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:58.764786 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173c888-7cab-489c-aaa5-b755e20302a3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:58.775829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 183/377] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Jul 19 15:16:58 2022] GET /identity/v3/role_assignments?group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445&scope.project.id=2bda471206fe4fbb88370df9ab3bdfac&role.id=44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:58.798811 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:58.802231 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebb573-f547-41ec-bd24-d8910d4096f1 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:58.865856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a2191-1dbc-456f-8ec6-c53eced0c004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:58.866758 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a2191-1dbc-456f-8ec6-c53eced0c004 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:58.867046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a2191-1dbc-456f-8ec6-c53eced0c004 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:58.867328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a2191-1dbc-456f-8ec6-c53eced0c004 None admin] PATH_INFO: `/v3/projects/5bb6648f966d4104a0136e3bb203cb76` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:58.870043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a2191-1dbc-456f-8ec6-c53eced0c004 None admin] RBAC: Authorizing `identity:get_project(project_id=5bb6648f966d4104a0136e3bb203cb76)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:58.871569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a2191-1dbc-456f-8ec6-c53eced0c004 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:58.873852 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 195/378] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:16:58 2022] GET /identity/v3/projects/5bb6648f966d4104a0136e3bb203cb76 => generated 344 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:58.879516 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173c888-7cab-489c-aaa5-b755e20302a3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:59.019807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173c888-7cab-489c-aaa5-b755e20302a3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:59.042668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:59.071633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b31ea-8d09-466e-94e6-789f513d1bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:59.072688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b31ea-8d09-466e-94e6-789f513d1bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:59.072976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b31ea-8d09-466e-94e6-789f513d1bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:59.073283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b31ea-8d09-466e-94e6-789f513d1bb5 None admin] PATH_INFO: `/v3/groups/2c7e2c2a6d83453bb6c63f3f7854fa03` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:59.081900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b31ea-8d09-466e-94e6-789f513d1bb5 None admin] RBAC: Authorizing `identity:get_group(group_id=2c7e2c2a6d83453bb6c63f3f7854fa03)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:59.084129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b31ea-8d09-466e-94e6-789f513d1bb5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:59.085915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 184/379] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Jul 19 15:16:58 2022] GET /identity/v3/groups/2c7e2c2a6d83453bb6c63f3f7854fa03 => generated 229 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:59.091869 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a2191-1dbc-456f-8ec6-c53eced0c004 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:59.215978 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a2191-1dbc-456f-8ec6-c53eced0c004 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:16:59.257118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674129f-dd02-42ed-9cb9-40066d1f331b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:16:59.257825 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674129f-dd02-42ed-9cb9-40066d1f331b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:59.258111 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674129f-dd02-42ed-9cb9-40066d1f331b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:59.258393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674129f-dd02-42ed-9cb9-40066d1f331b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:59.259250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674129f-dd02-42ed-9cb9-40066d1f331b None admin] RBAC: Adding query filter params (group.id=2c7e2c2a6d83453bb6c63f3f7854fa03, role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=5bb6648f966d4104a0136e3bb203cb76) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:16:59.259569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674129f-dd02-42ed-9cb9-40066d1f331b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:16:59.261032 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674129f-dd02-42ed-9cb9-40066d1f331b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:16:59.269905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 196/380] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Jul 19 15:16:59 2022] GET /identity/v3/role_assignments?group.id=2c7e2c2a6d83453bb6c63f3f7854fa03&scope.project.id=5bb6648f966d4104a0136e3bb203cb76&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:16:59.699851 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59bf29-305f-48c7-afe4-81ef4a0326dd None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:59.700201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59bf29-305f-48c7-afe4-81ef4a0326dd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:59.700403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59bf29-305f-48c7-afe4-81ef4a0326dd None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:59.701894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 185/381] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:16:59.707251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cba59-fe3a-4a80-a697-d39dfab47322 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:16:59.707553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cba59-fe3a-4a80-a697-d39dfab47322 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:16:59.707838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cba59-fe3a-4a80-a697-d39dfab47322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:16:59.729281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-826cba59-fe3a-4a80-a697-d39dfab47322 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:16:59.730488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-826cba59-fe3a-4a80-a697-d39dfab47322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:16:59.767701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:16:59.826821 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 197/382] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:16:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:16:59.867319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59bf29-305f-48c7-afe4-81ef4a0326dd None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:16:59.996739 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59bf29-305f-48c7-afe4-81ef4a0326dd None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:00.042061 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a76cc8-a2a3-4289-8354-05dbbe51f813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:00.042936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a76cc8-a2a3-4289-8354-05dbbe51f813 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:00.043226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a76cc8-a2a3-4289-8354-05dbbe51f813 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:00.043507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a76cc8-a2a3-4289-8354-05dbbe51f813 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:00.056244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a76cc8-a2a3-4289-8354-05dbbe51f813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-05a76cc8-a2a3-4289-8354-05dbbe51f813 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:00.064268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:00.065914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:00.066810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:00.066810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:00.066810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:00.066810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:00.066810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:00.066810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:00.066810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:00.066810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:00.066810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:00.066810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:00.066810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:17:00.066810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:00.069592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 186/383] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:16:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:00.071717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cba59-fe3a-4a80-a697-d39dfab47322 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:00.177898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cba59-fe3a-4a80-a697-d39dfab47322 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:00.215391 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f49e4-52a1-4ac4-a157-f23cde6bcc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:00.216071 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f49e4-52a1-4ac4-a157-f23cde6bcc34 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:00.216356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f49e4-52a1-4ac4-a157-f23cde6bcc34 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:00.216635 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f49e4-52a1-4ac4-a157-f23cde6bcc34 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:00.217423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f49e4-52a1-4ac4-a157-f23cde6bcc34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:00.217733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f49e4-52a1-4ac4-a157-f23cde6bcc34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:00.219111 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f49e4-52a1-4ac4-a157-f23cde6bcc34 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:00.228548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 198/384] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:00.234035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a76cc8-a2a3-4289-8354-05dbbe51f813 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:00.276678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:00.351588 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a76cc8-a2a3-4289-8354-05dbbe51f813 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:00.388006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413a5b5-eaf5-4a22-b462-7145dbc34073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:00.388867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413a5b5-eaf5-4a22-b462-7145dbc34073 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:00.389163 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413a5b5-eaf5-4a22-b462-7145dbc34073 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:00.389454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413a5b5-eaf5-4a22-b462-7145dbc34073 None admin] PATH_INFO: `/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:00.390633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413a5b5-eaf5-4a22-b462-7145dbc34073 None admin] RBAC: Authorizing `identity:get_group(group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:00.392086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413a5b5-eaf5-4a22-b462-7145dbc34073 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:00.393744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 187/385] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Jul 19 15:17:00 2022] GET /identity/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:00.399799 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f49e4-52a1-4ac4-a157-f23cde6bcc34 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:00.426736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:00.504582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f49e4-52a1-4ac4-a157-f23cde6bcc34 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:00.540514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2a787-ee5e-46d5-951d-40acd65a503d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:00.541401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2a787-ee5e-46d5-951d-40acd65a503d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:00.541684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2a787-ee5e-46d5-951d-40acd65a503d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:00.541959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2a787-ee5e-46d5-951d-40acd65a503d None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:00.543162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2a787-ee5e-46d5-951d-40acd65a503d None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:00.544607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2a787-ee5e-46d5-951d-40acd65a503d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:00.546638 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 199/386] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:17:00 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:00.552759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413a5b5-eaf5-4a22-b462-7145dbc34073 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:00.710715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413a5b5-eaf5-4a22-b462-7145dbc34073 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:00.722992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:00.766810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f184e-e45e-41f5-aa1a-cab8716cae84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:00.767620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f184e-e45e-41f5-aa1a-cab8716cae84 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:00.767899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f184e-e45e-41f5-aa1a-cab8716cae84 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:00.768173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f184e-e45e-41f5-aa1a-cab8716cae84 None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445/roles/44bd561cc5594ffd9bcc27ef6dad0fc5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:00.770443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f184e-e45e-41f5-aa1a-cab8716cae84 None admin] RBAC: Authorizing `identity:create_grant(project_id=30d54bfc38f04330a2dc73fcb0219ec5, group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445, role_id=44bd561cc5594ffd9bcc27ef6dad0fc5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:00.771920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f184e-e45e-41f5-aa1a-cab8716cae84 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:00.789461 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 188/387] 10.222.0.42 () {58 vars in 1442 bytes} [Tue Jul 19 15:17:00 2022] PUT /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445/roles/44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:17:01.018912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fbd20-9472-4490-800a-b54eb74ae03e None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:01.018912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fbd20-9472-4490-800a-b54eb74ae03e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:01.018912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fbd20-9472-4490-800a-b54eb74ae03e None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:01.020182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 200/388] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:01.028677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd30bc-582b-4a10-98be-a55ab19188f7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:01.028677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd30bc-582b-4a10-98be-a55ab19188f7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:01.028677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd30bc-582b-4a10-98be-a55ab19188f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:01.046010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-acbd30bc-582b-4a10-98be-a55ab19188f7 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:01.047533 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-acbd30bc-582b-4a10-98be-a55ab19188f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:01.125289 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 189/389] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:01.141707 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fbd20-9472-4490-800a-b54eb74ae03e None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:01.240644 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:01.261701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fbd20-9472-4490-800a-b54eb74ae03e None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:01.277962 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa73e55-f69b-481e-a846-6dbac5eb64cf None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:01.278222 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa73e55-f69b-481e-a846-6dbac5eb64cf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:01.278436 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa73e55-f69b-481e-a846-6dbac5eb64cf None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:01.279811 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 190/390] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:17:01.286251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747be74-1e0e-46ad-aab9-aa181ed83d79 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:01.287268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747be74-1e0e-46ad-aab9-aa181ed83d79 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:01.287737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747be74-1e0e-46ad-aab9-aa181ed83d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:01.314362 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b747be74-1e0e-46ad-aab9-aa181ed83d79 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:01.316012 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b747be74-1e0e-46ad-aab9-aa181ed83d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:01.335881 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c6a75-38f1-446b-bbc0-dafb5f3cbcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:01.336694 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c6a75-38f1-446b-bbc0-dafb5f3cbcf1 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:01.336985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c6a75-38f1-446b-bbc0-dafb5f3cbcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:01.337284 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c6a75-38f1-446b-bbc0-dafb5f3cbcf1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:01.359334 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c6a75-38f1-446b-bbc0-dafb5f3cbcf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-202c6a75-38f1-446b-bbc0-dafb5f3cbcf1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:01.367748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:01.368731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:01.369918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:01.369918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:01.369918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:01.369918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:01.369918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:01.369918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:01.369918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:01.369918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:01.369918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:01.369918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:01.369918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:17:01.369918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:01.374749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 201/391] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:01.380567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c6a75-38f1-446b-bbc0-dafb5f3cbcf1 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:01.457330 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 191/392] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:01.486553 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747be74-1e0e-46ad-aab9-aa181ed83d79 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:01.532517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:01.580892 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c6a75-38f1-446b-bbc0-dafb5f3cbcf1 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:01.628005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a316dfa-a2e0-43c6-b28d-9fe274835be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:01.628683 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a316dfa-a2e0-43c6-b28d-9fe274835be3 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:01.628969 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a316dfa-a2e0-43c6-b28d-9fe274835be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:01.629301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a316dfa-a2e0-43c6-b28d-9fe274835be3 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:01.630666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a316dfa-a2e0-43c6-b28d-9fe274835be3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:01.631480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a316dfa-a2e0-43c6-b28d-9fe274835be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:01.632872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a316dfa-a2e0-43c6-b28d-9fe274835be3 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:01.639155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747be74-1e0e-46ad-aab9-aa181ed83d79 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:01.653051 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 202/393] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:17:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:01.655217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d45bfc-80ef-4ee8-bfd3-917058c36742 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:01.655655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d45bfc-80ef-4ee8-bfd3-917058c36742 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:01.656072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d45bfc-80ef-4ee8-bfd3-917058c36742 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:01.658218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 203/394] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:01.661263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d45bfc-80ef-4ee8-bfd3-917058c36742 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:01.683142 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a500c-fb86-4f56-a3c0-2bbf395b2e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:01.684085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a500c-fb86-4f56-a3c0-2bbf395b2e45 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:01.684484 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a500c-fb86-4f56-a3c0-2bbf395b2e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:01.684855 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a500c-fb86-4f56-a3c0-2bbf395b2e45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:01.686796 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:01.698089 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a500c-fb86-4f56-a3c0-2bbf395b2e45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-a47a500c-fb86-4f56-a3c0-2bbf395b2e45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:01.706512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:01.707402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:01.708327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:01.708327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:01.708327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:01.708327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:01.708327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:01.708327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:01.708327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:01.708327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:01.708327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:01.708327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:01.708327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:17:01.708327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:01.708327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 192/395] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:01.708995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b068e59-e818-4eae-b4e0-f3244a577fb8 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:01.711566 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b068e59-e818-4eae-b4e0-f3244a577fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:01.711566 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b068e59-e818-4eae-b4e0-f3244a577fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:01.727610 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8b068e59-e818-4eae-b4e0-f3244a577fb8 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:01.729052 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8b068e59-e818-4eae-b4e0-f3244a577fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:01.790998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d45bfc-80ef-4ee8-bfd3-917058c36742 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:01.809747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 193/396] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:01.816850 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b068e59-e818-4eae-b4e0-f3244a577fb8 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:01.832644 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec139169-c136-4790-a5e5-d1a64c8c3734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:01.834404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec139169-c136-4790-a5e5-d1a64c8c3734 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:01.834838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec139169-c136-4790-a5e5-d1a64c8c3734 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:01.835637 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec139169-c136-4790-a5e5-d1a64c8c3734 None admin] PATH_INFO: `/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:01.836995 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec139169-c136-4790-a5e5-d1a64c8c3734 None admin] RBAC: Authorizing `identity:get_group(group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:01.838595 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec139169-c136-4790-a5e5-d1a64c8c3734 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:01.845374 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 204/397] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Jul 19 15:17:01 2022] GET /identity/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:01.847167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec139169-c136-4790-a5e5-d1a64c8c3734 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:01.852148 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:01.955459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:01.970763 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b068e59-e818-4eae-b4e0-f3244a577fb8 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:01.978960 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec139169-c136-4790-a5e5-d1a64c8c3734 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:02.009198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabadf12-35ef-4fca-b0c5-f5f8f77ef7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:02.009877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabadf12-35ef-4fca-b0c5-f5f8f77ef7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:02.010181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabadf12-35ef-4fca-b0c5-f5f8f77ef7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:02.010432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabadf12-35ef-4fca-b0c5-f5f8f77ef7b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:02.011174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabadf12-35ef-4fca-b0c5-f5f8f77ef7b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:02.011485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabadf12-35ef-4fca-b0c5-f5f8f77ef7b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:02.012967 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabadf12-35ef-4fca-b0c5-f5f8f77ef7b5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:02.019850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781af68-b2b0-49e3-82af-106f72fb1d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:02.021937 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781af68-b2b0-49e3-82af-106f72fb1d59 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:02.023016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781af68-b2b0-49e3-82af-106f72fb1d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:02.023541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781af68-b2b0-49e3-82af-106f72fb1d59 None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:02.025567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781af68-b2b0-49e3-82af-106f72fb1d59 None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:02.027291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781af68-b2b0-49e3-82af-106f72fb1d59 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:02.029181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 205/398] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:17:01 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:02.034019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781af68-b2b0-49e3-82af-106f72fb1d59 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:02.034131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 194/399] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:17:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:02.034131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abee76c-b702-4f20-8001-7b88ee3c27a7 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:02.034131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abee76c-b702-4f20-8001-7b88ee3c27a7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:02.034131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abee76c-b702-4f20-8001-7b88ee3c27a7 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:02.034714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 195/400] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:02.035088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abee76c-b702-4f20-8001-7b88ee3c27a7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:02.124226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:02.146298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781af68-b2b0-49e3-82af-106f72fb1d59 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:02.147704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abee76c-b702-4f20-8001-7b88ee3c27a7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:02.185339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9dc19-25fe-4e56-80f4-e61f643c2293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:02.186187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9dc19-25fe-4e56-80f4-e61f643c2293 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:02.186458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9dc19-25fe-4e56-80f4-e61f643c2293 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:02.186734 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9dc19-25fe-4e56-80f4-e61f643c2293 None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:02.186852 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7ef3e-286f-4609-9282-2762ff2429a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:02.187624 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7ef3e-286f-4609-9282-2762ff2429a1 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:02.187906 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7ef3e-286f-4609-9282-2762ff2429a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:02.188200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7ef3e-286f-4609-9282-2762ff2429a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:02.188834 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9dc19-25fe-4e56-80f4-e61f643c2293 None admin] RBAC: Authorizing `identity:create_grant(project_id=30d54bfc38f04330a2dc73fcb0219ec5, group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:02.190282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9dc19-25fe-4e56-80f4-e61f643c2293 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:02.199724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7ef3e-286f-4609-9282-2762ff2429a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:02.204877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 196/401] 10.222.0.42 () {58 vars in 1442 bytes} [Tue Jul 19 15:17:02 2022] PUT /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:17:02.206041 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9dc19-25fe-4e56-80f4-e61f643c2293 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-e3a7ef3e-286f-4609-9282-2762ff2429a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:02.208215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:02.209121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:02.210463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:02.210463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:02.210463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:02.210463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:02.210463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:02.210463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:02.210463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:02.210463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:02.210463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:02.210463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:02.210463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:17:02.210463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:02.210463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 206/402] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:17:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:02.211327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0943c2-9258-45df-8bc4-7f460d5d5209 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:02.211566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0943c2-9258-45df-8bc4-7f460d5d5209 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:02.211821 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0943c2-9258-45df-8bc4-7f460d5d5209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:02.234640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-af0943c2-9258-45df-8bc4-7f460d5d5209 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:02.235974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-af0943c2-9258-45df-8bc4-7f460d5d5209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:02.241048 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:02.343815 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 207/403] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:02.345171 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0943c2-9258-45df-8bc4-7f460d5d5209 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:02.365589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9dc19-25fe-4e56-80f4-e61f643c2293 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:02.413638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86911330-ee93-49a0-9a06-e295ecccbaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:02.414121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86911330-ee93-49a0-9a06-e295ecccbaac None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:02.414405 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86911330-ee93-49a0-9a06-e295ecccbaac None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:02.414687 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86911330-ee93-49a0-9a06-e295ecccbaac None admin] PATH_INFO: `/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:02.415927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86911330-ee93-49a0-9a06-e295ecccbaac None admin] RBAC: Authorizing `identity:get_group(group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:02.419397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86911330-ee93-49a0-9a06-e295ecccbaac None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:02.419397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 197/404] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Jul 19 15:17:02 2022] GET /identity/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:02.420195 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86911330-ee93-49a0-9a06-e295ecccbaac None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:02.436053 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:02.480542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0943c2-9258-45df-8bc4-7f460d5d5209 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:02.525229 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0339f-4de6-4357-9463-1aeba0f89e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:02.525909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0339f-4de6-4357-9463-1aeba0f89e6d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:02.526227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0339f-4de6-4357-9463-1aeba0f89e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:02.527179 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0339f-4de6-4357-9463-1aeba0f89e6d None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:02.528443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0339f-4de6-4357-9463-1aeba0f89e6d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:02.528794 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0339f-4de6-4357-9463-1aeba0f89e6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:02.530732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0339f-4de6-4357-9463-1aeba0f89e6d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:02.543075 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 208/405] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:02.543963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0339f-4de6-4357-9463-1aeba0f89e6d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:02.546612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:02.560779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86911330-ee93-49a0-9a06-e295ecccbaac None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:02.609819 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f518885-648a-48a6-b152-b5f34ad88984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:02.610885 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f518885-648a-48a6-b152-b5f34ad88984 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:02.611203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f518885-648a-48a6-b152-b5f34ad88984 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:02.611856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f518885-648a-48a6-b152-b5f34ad88984 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:02.626271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f518885-648a-48a6-b152-b5f34ad88984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9f518885-648a-48a6-b152-b5f34ad88984 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:02.637329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:02.638263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:02.639071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:02.639071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:02.639071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:02.639071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:02.639071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:02.639071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:02.639071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:02.639071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:02.639071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:02.639071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:02.639071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:02.639071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:02.641224 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 198/406] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:17:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:02.642057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f518885-648a-48a6-b152-b5f34ad88984 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:02.661807 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0339f-4de6-4357-9463-1aeba0f89e6d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:02.709725 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df3262-fb42-473d-af19-7e77efac7b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:02.711176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df3262-fb42-473d-af19-7e77efac7b9f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:02.711465 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df3262-fb42-473d-af19-7e77efac7b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:02.711749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df3262-fb42-473d-af19-7e77efac7b9f None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:02.712979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df3262-fb42-473d-af19-7e77efac7b9f None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:02.713467 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:02.714778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df3262-fb42-473d-af19-7e77efac7b9f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:02.721399 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 209/407] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:17:02 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:02.722541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df3262-fb42-473d-af19-7e77efac7b9f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:02.762881 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f518885-648a-48a6-b152-b5f34ad88984 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:02.804334 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be56d5-0729-430b-82d9-94bfbc8233a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:02.805227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be56d5-0729-430b-82d9-94bfbc8233a4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:02.805508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be56d5-0729-430b-82d9-94bfbc8233a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:02.805779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be56d5-0729-430b-82d9-94bfbc8233a4 None admin] PATH_INFO: `/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:02.806963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be56d5-0729-430b-82d9-94bfbc8233a4 None admin] RBAC: Authorizing `identity:get_group(group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:02.808412 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be56d5-0729-430b-82d9-94bfbc8233a4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:02.809990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:02.815207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 199/408] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Jul 19 15:17:02 2022] GET /identity/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:02.816054 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be56d5-0729-430b-82d9-94bfbc8233a4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:02.855942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df3262-fb42-473d-af19-7e77efac7b9f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:02.904522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e448d4b-4cdc-4a09-a5a8-6b4bd082b0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:02.906239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e448d4b-4cdc-4a09-a5a8-6b4bd082b0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:02.906734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e448d4b-4cdc-4a09-a5a8-6b4bd082b0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:02.907435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e448d4b-4cdc-4a09-a5a8-6b4bd082b0b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:02.908629 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e448d4b-4cdc-4a09-a5a8-6b4bd082b0b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:02.909540 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e448d4b-4cdc-4a09-a5a8-6b4bd082b0b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:02.911206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e448d4b-4cdc-4a09-a5a8-6b4bd082b0b6 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:02.928679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be56d5-0729-430b-82d9-94bfbc8233a4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:02.929484 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 210/409] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:02.931513 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e448d4b-4cdc-4a09-a5a8-6b4bd082b0b6 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:02.965051 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e9db2-93c3-4fc7-a485-dd73c26d7353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:02.965879 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e9db2-93c3-4fc7-a485-dd73c26d7353 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:02.966162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e9db2-93c3-4fc7-a485-dd73c26d7353 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:02.966439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e9db2-93c3-4fc7-a485-dd73c26d7353 None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:02.968426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e9db2-93c3-4fc7-a485-dd73c26d7353 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bda471206fe4fbb88370df9ab3bdfac, group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:02.970010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e9db2-93c3-4fc7-a485-dd73c26d7353 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:02.972445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:02.987993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 200/410] 10.222.0.42 () {58 vars in 1442 bytes} [Tue Jul 19 15:17:02 2022] PUT /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:17:02.994107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e9db2-93c3-4fc7-a485-dd73c26d7353 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:03.053640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e448d4b-4cdc-4a09-a5a8-6b4bd082b0b6 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:03.087759 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:03.095502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015223d1-9e82-4c95-9c10-75b24f0704ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:03.096992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015223d1-9e82-4c95-9c10-75b24f0704ae None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:03.097530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015223d1-9e82-4c95-9c10-75b24f0704ae None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:03.098224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015223d1-9e82-4c95-9c10-75b24f0704ae None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:03.100004 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015223d1-9e82-4c95-9c10-75b24f0704ae None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:03.101705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015223d1-9e82-4c95-9c10-75b24f0704ae None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:03.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 211/411] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:17:02 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:03.111721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015223d1-9e82-4c95-9c10-75b24f0704ae None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:03.190392 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e9db2-93c3-4fc7-a485-dd73c26d7353 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:03.260237 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01fb7f-a1a0-4279-b6a9-0707e2562a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:03.260932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01fb7f-a1a0-4279-b6a9-0707e2562a79 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:03.261106 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01fb7f-a1a0-4279-b6a9-0707e2562a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:03.262306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01fb7f-a1a0-4279-b6a9-0707e2562a79 None admin] PATH_INFO: `/v3/groups/2c7e2c2a6d83453bb6c63f3f7854fa03` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:03.264746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01fb7f-a1a0-4279-b6a9-0707e2562a79 None admin] RBAC: Authorizing `identity:get_group(group_id=2c7e2c2a6d83453bb6c63f3f7854fa03)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:03.266252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01fb7f-a1a0-4279-b6a9-0707e2562a79 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:03.268479 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 201/412] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Jul 19 15:17:02 2022] GET /identity/v3/groups/2c7e2c2a6d83453bb6c63f3f7854fa03 => generated 229 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:03.269648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e5ea9-3ab2-473b-8a95-ccf67d6fa8d7 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:03.269938 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e5ea9-3ab2-473b-8a95-ccf67d6fa8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:03.270218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e5ea9-3ab2-473b-8a95-ccf67d6fa8d7 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:03.271764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 202/413] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:03.273979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e5ea9-3ab2-473b-8a95-ccf67d6fa8d7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:03.304429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015223d1-9e82-4c95-9c10-75b24f0704ae None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:03.348766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:03.351377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421f8dc-773e-4580-97dc-aff749891e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:03.352643 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421f8dc-773e-4580-97dc-aff749891e5e None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:03.352928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421f8dc-773e-4580-97dc-aff749891e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:03.353233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421f8dc-773e-4580-97dc-aff749891e5e None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445/roles/44bd561cc5594ffd9bcc27ef6dad0fc5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:03.355563 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421f8dc-773e-4580-97dc-aff749891e5e None admin] RBAC: Authorizing `identity:create_grant(project_id=2bda471206fe4fbb88370df9ab3bdfac, group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445, role_id=44bd561cc5594ffd9bcc27ef6dad0fc5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:03.357373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421f8dc-773e-4580-97dc-aff749891e5e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:03.374801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 212/414] 10.222.0.42 () {58 vars in 1442 bytes} [Tue Jul 19 15:17:03 2022] PUT /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445/roles/44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:17:03.378352 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bb3c2-3719-4183-8c13-a89ea902eb8d None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:03.378899 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bb3c2-3719-4183-8c13-a89ea902eb8d None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:03.379421 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bb3c2-3719-4183-8c13-a89ea902eb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:03.414731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-b91bb3c2-3719-4183-8c13-a89ea902eb8d None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:03.416336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b91bb3c2-3719-4183-8c13-a89ea902eb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:03.441947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e5ea9-3ab2-473b-8a95-ccf67d6fa8d7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:03.510939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:03.512270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9f53b-b350-483e-bf3a-e4c85b0c9115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:03.514243 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9f53b-b350-483e-bf3a-e4c85b0c9115 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:03.514762 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9f53b-b350-483e-bf3a-e4c85b0c9115 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:03.515914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9f53b-b350-483e-bf3a-e4c85b0c9115 None admin] PATH_INFO: `/v3/projects/5bb6648f966d4104a0136e3bb203cb76` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:03.518771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9f53b-b350-483e-bf3a-e4c85b0c9115 None admin] RBAC: Authorizing `identity:get_project(project_id=5bb6648f966d4104a0136e3bb203cb76)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:03.522754 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9f53b-b350-483e-bf3a-e4c85b0c9115 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:03.527555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 203/415] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:17:03 2022] GET /identity/v3/projects/5bb6648f966d4104a0136e3bb203cb76 => generated 344 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:03.532149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 213/416] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:03.533331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9f53b-b350-483e-bf3a-e4c85b0c9115 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:03.574405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bb3c2-3719-4183-8c13-a89ea902eb8d None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:03.678115 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9f53b-b350-483e-bf3a-e4c85b0c9115 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:03.712603 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:03.720691 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a45bd6-73b3-4ced-b5b9-d47af78e678a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:03.721886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a45bd6-73b3-4ced-b5b9-d47af78e678a None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:03.722172 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a45bd6-73b3-4ced-b5b9-d47af78e678a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:03.722452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a45bd6-73b3-4ced-b5b9-d47af78e678a None admin] PATH_INFO: `/v3/projects/5bb6648f966d4104a0136e3bb203cb76/groups/2c7e2c2a6d83453bb6c63f3f7854fa03/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:03.724161 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bb3c2-3719-4183-8c13-a89ea902eb8d None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:03.725155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a45bd6-73b3-4ced-b5b9-d47af78e678a None admin] RBAC: Authorizing `identity:create_grant(project_id=5bb6648f966d4104a0136e3bb203cb76, group_id=2c7e2c2a6d83453bb6c63f3f7854fa03, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:03.726880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a45bd6-73b3-4ced-b5b9-d47af78e678a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:03.745445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 204/417] 10.222.0.42 () {58 vars in 1442 bytes} [Tue Jul 19 15:17:03 2022] PUT /identity/v3/projects/5bb6648f966d4104a0136e3bb203cb76/groups/2c7e2c2a6d83453bb6c63f3f7854fa03/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:17:03.771359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e62c18-28be-4fa0-a9b5-03f7b35e46e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:03.772696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e62c18-28be-4fa0-a9b5-03f7b35e46e7 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:03.773204 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e62c18-28be-4fa0-a9b5-03f7b35e46e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:03.773772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e62c18-28be-4fa0-a9b5-03f7b35e46e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:03.787653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e62c18-28be-4fa0-a9b5-03f7b35e46e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-a7e62c18-28be-4fa0-a9b5-03f7b35e46e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:03.797128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:03.798190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:03.799030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:03.799030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:03.799030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:03.799030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:03.799030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:03.799030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:03.799030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:03.799030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:03.799030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:03.799030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:03.799030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:17:03.799030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:03.801883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 214/418] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:17:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:03.809256 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a45bd6-73b3-4ced-b5b9-d47af78e678a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:03.956913 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a45bd6-73b3-4ced-b5b9-d47af78e678a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:03.994678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7d0e3-6bb8-47c2-91c4-9c1dc7b75cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:03.995368 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7d0e3-6bb8-47c2-91c4-9c1dc7b75cae None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:03.995659 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7d0e3-6bb8-47c2-91c4-9c1dc7b75cae None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:03.995942 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7d0e3-6bb8-47c2-91c4-9c1dc7b75cae None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:03.996781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7d0e3-6bb8-47c2-91c4-9c1dc7b75cae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:03.997100 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7d0e3-6bb8-47c2-91c4-9c1dc7b75cae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:03.998638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7d0e3-6bb8-47c2-91c4-9c1dc7b75cae None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:04.012160 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 205/419] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:04.017364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e62c18-28be-4fa0-a9b5-03f7b35e46e7 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:04.080175 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:04.134689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e62c18-28be-4fa0-a9b5-03f7b35e46e7 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:04.170303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6cd94-1525-46bd-af8a-dc806b5c54fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:04.171350 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6cd94-1525-46bd-af8a-dc806b5c54fb None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:04.171853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6cd94-1525-46bd-af8a-dc806b5c54fb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:04.172327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6cd94-1525-46bd-af8a-dc806b5c54fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:04.256513 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6cd94-1525-46bd-af8a-dc806b5c54fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:04.258407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6cd94-1525-46bd-af8a-dc806b5c54fb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:04.309599 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:04.363486 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 215/420] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 15:17:04.374261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7d0e3-6bb8-47c2-91c4-9c1dc7b75cae None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:04.462027 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:04.517233 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7d0e3-6bb8-47c2-91c4-9c1dc7b75cae None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:04.556625 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a1b68-cd6b-4d02-9bea-4ffc44ee67bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:04.557695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a1b68-cd6b-4d02-9bea-4ffc44ee67bc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:04.557983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a1b68-cd6b-4d02-9bea-4ffc44ee67bc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:04.558274 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a1b68-cd6b-4d02-9bea-4ffc44ee67bc None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:04.559834 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a1b68-cd6b-4d02-9bea-4ffc44ee67bc None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:04.560918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e76cc6-845c-41b9-bf6e-edd800aa4f50 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:04.561233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e76cc6-845c-41b9-bf6e-edd800aa4f50 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:04.561337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a1b68-cd6b-4d02-9bea-4ffc44ee67bc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:04.561509 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e76cc6-845c-41b9-bf6e-edd800aa4f50 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:04.563155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 216/422] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:04.563273 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 206/422] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:17:04 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:04.570294 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e76cc6-845c-41b9-bf6e-edd800aa4f50 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:04.574512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93caeb4-3139-476e-8912-e35234fc0832 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:04.574801 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93caeb4-3139-476e-8912-e35234fc0832 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:04.575077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93caeb4-3139-476e-8912-e35234fc0832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:04.602097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f93caeb4-3139-476e-8912-e35234fc0832 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:04.603272 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f93caeb4-3139-476e-8912-e35234fc0832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:04.714287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e76cc6-845c-41b9-bf6e-edd800aa4f50 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:04.732369 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 207/423] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 19 15:17:04.762780 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206c00c-031e-432c-94b8-917cbc56b994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:04.763700 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206c00c-031e-432c-94b8-917cbc56b994 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:04.763986 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206c00c-031e-432c-94b8-917cbc56b994 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:04.764266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206c00c-031e-432c-94b8-917cbc56b994 None admin] PATH_INFO: `/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:04.765528 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206c00c-031e-432c-94b8-917cbc56b994 None admin] RBAC: Authorizing `identity:get_group(group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:04.767076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206c00c-031e-432c-94b8-917cbc56b994 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:04.769120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 217/424] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Jul 19 15:17:04 2022] GET /identity/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:04.775811 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93caeb4-3139-476e-8912-e35234fc0832 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:04.790581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206c00c-031e-432c-94b8-917cbc56b994 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:04.812904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:04.880002 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:04.956642 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206c00c-031e-432c-94b8-917cbc56b994 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:04.978587 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93caeb4-3139-476e-8912-e35234fc0832 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:05.005476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce388a2-8221-47f7-b1da-562b6ce90674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:05.007914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce388a2-8221-47f7-b1da-562b6ce90674 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:05.009918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce388a2-8221-47f7-b1da-562b6ce90674 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:05.010361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce388a2-8221-47f7-b1da-562b6ce90674 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:05.033303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce388a2-8221-47f7-b1da-562b6ce90674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-0ce388a2-8221-47f7-b1da-562b6ce90674 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:05.053615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:05.054806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:05.055853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:05.055853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:05.055853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:05.055853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:05.055853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:05.055853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:05.055853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:05.055853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:05.055853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:05.055853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:05.055853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:17:05.055853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:05.060608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43a426-98b3-438d-ad9e-fa0278aa796e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:05.061015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 218/425] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:05.065775 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43a426-98b3-438d-ad9e-fa0278aa796e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:05.066274 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43a426-98b3-438d-ad9e-fa0278aa796e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:05.066702 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43a426-98b3-438d-ad9e-fa0278aa796e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:05.068545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43a426-98b3-438d-ad9e-fa0278aa796e None admin] RBAC: Adding query filter params (group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445, role.id=44bd561cc5594ffd9bcc27ef6dad0fc5, scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:05.069259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43a426-98b3-438d-ad9e-fa0278aa796e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:05.071011 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43a426-98b3-438d-ad9e-fa0278aa796e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:05.075767 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce388a2-8221-47f7-b1da-562b6ce90674 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:05.091155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 208/426] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Jul 19 15:17:04 2022] GET /identity/v3/role_assignments?group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445&scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5&role.id=44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 611 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:05.200942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:05.237320 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce388a2-8221-47f7-b1da-562b6ce90674 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:05.295478 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45356ca-06e9-4b1a-bd82-33bdfd427c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:05.296163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45356ca-06e9-4b1a-bd82-33bdfd427c6f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:05.296445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45356ca-06e9-4b1a-bd82-33bdfd427c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:05.296720 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45356ca-06e9-4b1a-bd82-33bdfd427c6f None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:05.297511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45356ca-06e9-4b1a-bd82-33bdfd427c6f None admin] RBAC: Adding query filter params (name=member) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:05.297822 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45356ca-06e9-4b1a-bd82-33bdfd427c6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:05.299226 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45356ca-06e9-4b1a-bd82-33bdfd427c6f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:05.313765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 219/427] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:17:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:05.320773 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43a426-98b3-438d-ad9e-fa0278aa796e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:05.408275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92b4f8-5688-44a2-be67-af11bcc7e628 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:05.408622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92b4f8-5688-44a2-be67-af11bcc7e628 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:05.408827 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92b4f8-5688-44a2-be67-af11bcc7e628 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:05.410269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 220/428] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:17:05.422299 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b9c10-a268-429b-85c8-5c4e380919ae None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:05.422587 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b9c10-a268-429b-85c8-5c4e380919ae None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:05.422868 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b9c10-a268-429b-85c8-5c4e380919ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:05.447848 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ea1b9c10-a268-429b-85c8-5c4e380919ae None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:05.450095 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ea1b9c10-a268-429b-85c8-5c4e380919ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:05.456011 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43a426-98b3-438d-ad9e-fa0278aa796e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:05.471539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:05.498849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b031b-c4d9-47b0-b28d-1854ce60e8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:05.499542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b031b-c4d9-47b0-b28d-1854ce60e8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:05.499825 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b031b-c4d9-47b0-b28d-1854ce60e8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:05.500103 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b031b-c4d9-47b0-b28d-1854ce60e8e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:05.552004 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 221/429] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:17:05.576388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b9c10-a268-429b-85c8-5c4e380919ae None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:05.580751 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b031b-c4d9-47b0-b28d-1854ce60e8e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:05.582239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b031b-c4d9-47b0-b28d-1854ce60e8e4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:05.671923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:05.700646 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 209/430] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:17:05.706255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b031b-c4d9-47b0-b28d-1854ce60e8e4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:05.712070 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b9c10-a268-429b-85c8-5c4e380919ae None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:05.725206 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:05.766375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed439b-a75e-4174-b818-fb113e235afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:05.767659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed439b-a75e-4174-b818-fb113e235afa None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:05.768085 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed439b-a75e-4174-b818-fb113e235afa None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:05.768710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed439b-a75e-4174-b818-fb113e235afa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:05.783597 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed439b-a75e-4174-b818-fb113e235afa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-d4ed439b-a75e-4174-b818-fb113e235afa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:05.794375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:05.795480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:05.796443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:05.796443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:05.796443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:05.796443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:05.796443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:05.796443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:05.796443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:05.796443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:05.796443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:05.796443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:05.796443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 15:17:05.796443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:05.798907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 222/431] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:05.805034 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed439b-a75e-4174-b818-fb113e235afa None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:05.836628 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b031b-c4d9-47b0-b28d-1854ce60e8e4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:05.880456 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f0608-d48c-41ff-bdbe-24a6fe6fdfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:05.881565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f0608-d48c-41ff-bdbe-24a6fe6fdfe6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:05.881847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f0608-d48c-41ff-bdbe-24a6fe6fdfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:05.882657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f0608-d48c-41ff-bdbe-24a6fe6fdfe6 None admin] PATH_INFO: `/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:05.884123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f0608-d48c-41ff-bdbe-24a6fe6fdfe6 None admin] RBAC: Authorizing `identity:get_project(project_id=30d54bfc38f04330a2dc73fcb0219ec5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:05.886541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f0608-d48c-41ff-bdbe-24a6fe6fdfe6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:05.888612 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 210/432] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:17:05 2022] GET /identity/v3/projects/30d54bfc38f04330a2dc73fcb0219ec5 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:05.894002 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f0608-d48c-41ff-bdbe-24a6fe6fdfe6 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:05.921843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed439b-a75e-4174-b818-fb113e235afa None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:05.956449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:05.964130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190a11d-cc17-4216-a608-f912742fd577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:05.964904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190a11d-cc17-4216-a608-f912742fd577 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:05.965213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190a11d-cc17-4216-a608-f912742fd577 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:05.965593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190a11d-cc17-4216-a608-f912742fd577 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:05.966354 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190a11d-cc17-4216-a608-f912742fd577 None admin] RBAC: Adding query filter params (name=member) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:05.966663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190a11d-cc17-4216-a608-f912742fd577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:05.968215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190a11d-cc17-4216-a608-f912742fd577 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:05.978460 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 223/433] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:17:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:05.984611 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190a11d-cc17-4216-a608-f912742fd577 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:05.997200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:06.017002 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f0608-d48c-41ff-bdbe-24a6fe6fdfe6 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:06.053865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455492c3-7f04-43c4-a89d-d29231868ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:06.054744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455492c3-7f04-43c4-a89d-d29231868ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:06.055027 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455492c3-7f04-43c4-a89d-d29231868ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:06.055304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455492c3-7f04-43c4-a89d-d29231868ba2 None admin] PATH_INFO: `/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:06.056707 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455492c3-7f04-43c4-a89d-d29231868ba2 None admin] RBAC: Authorizing `identity:get_group(group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:06.058190 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455492c3-7f04-43c4-a89d-d29231868ba2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:06.060197 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 211/434] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Jul 19 15:17:05 2022] GET /identity/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:06.069546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455492c3-7f04-43c4-a89d-d29231868ba2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:06.099609 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190a11d-cc17-4216-a608-f912742fd577 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:06.138563 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23880310-2854-4a64-a534-37b6314e5f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:06.139218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23880310-2854-4a64-a534-37b6314e5f3c None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:06.139508 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23880310-2854-4a64-a534-37b6314e5f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:06.139783 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23880310-2854-4a64-a534-37b6314e5f3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:06.174404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455492c3-7f04-43c4-a89d-d29231868ba2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:06.205618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:06.215426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23880310-2854-4a64-a534-37b6314e5f3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:06.216217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63482db1-ab8d-45ba-a340-5fb2f5ac20f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:06.216893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23880310-2854-4a64-a534-37b6314e5f3c None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:06.217377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63482db1-ab8d-45ba-a340-5fb2f5ac20f9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:06.217909 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63482db1-ab8d-45ba-a340-5fb2f5ac20f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:06.218375 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63482db1-ab8d-45ba-a340-5fb2f5ac20f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:06.219574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63482db1-ab8d-45ba-a340-5fb2f5ac20f9 None admin] RBAC: Adding query filter params (group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445, role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f, scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:06.219952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63482db1-ab8d-45ba-a340-5fb2f5ac20f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:06.221487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63482db1-ab8d-45ba-a340-5fb2f5ac20f9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:06.235298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 212/435] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Jul 19 15:17:06 2022] GET /identity/v3/role_assignments?group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445&scope.project.id=30d54bfc38f04330a2dc73fcb0219ec5&role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:06.235298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7d53a-074d-4316-a02a-905141a8b0f3 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:06.235298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7d53a-074d-4316-a02a-905141a8b0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:06.235298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7d53a-074d-4316-a02a-905141a8b0f3 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:06.235298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 213/436] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:06.237805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cf464-cef7-4323-bf3a-79112f710c9e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:06.238093 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cf464-cef7-4323-bf3a-79112f710c9e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:06.240495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cf464-cef7-4323-bf3a-79112f710c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:06.258121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c72cf464-cef7-4323-bf3a-79112f710c9e None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:06.259619 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c72cf464-cef7-4323-bf3a-79112f710c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:06.271641 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:06.352017 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 224/437] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:17:06.353622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747defd8-67b6-4ee9-aca8-0590f8fcfdb4 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:06.353983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747defd8-67b6-4ee9-aca8-0590f8fcfdb4 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:06.354324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747defd8-67b6-4ee9-aca8-0590f8fcfdb4 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:06.356033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 225/438] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:06.357741 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747defd8-67b6-4ee9-aca8-0590f8fcfdb4 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:06.360985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 214/439] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:17:06.362483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917c388-0391-4527-8aa8-4c8f1178334a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:06.362782 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917c388-0391-4527-8aa8-4c8f1178334a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:06.363061 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917c388-0391-4527-8aa8-4c8f1178334a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:06.385426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f917c388-0391-4527-8aa8-4c8f1178334a None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:06.387169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f917c388-0391-4527-8aa8-4c8f1178334a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:06.465376 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 215/440] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:06.466337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917c388-0391-4527-8aa8-4c8f1178334a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:06.472916 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:06.487943 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747defd8-67b6-4ee9-aca8-0590f8fcfdb4 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:06.528728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb338b12-0c45-4f5c-888a-14b95c177efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:06.529634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb338b12-0c45-4f5c-888a-14b95c177efc None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:06.530003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb338b12-0c45-4f5c-888a-14b95c177efc None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:06.530343 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb338b12-0c45-4f5c-888a-14b95c177efc None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:06.531540 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb338b12-0c45-4f5c-888a-14b95c177efc None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:06.532977 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb338b12-0c45-4f5c-888a-14b95c177efc None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:06.534915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 226/441] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:17:06 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:06.535507 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb338b12-0c45-4f5c-888a-14b95c177efc None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:06.561181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:06.591150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917c388-0391-4527-8aa8-4c8f1178334a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:06.634953 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783457a8-e592-4f80-8a21-95b973cc82f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:06.635780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783457a8-e592-4f80-8a21-95b973cc82f9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:06.636067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783457a8-e592-4f80-8a21-95b973cc82f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:06.636345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783457a8-e592-4f80-8a21-95b973cc82f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:06.649245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783457a8-e592-4f80-8a21-95b973cc82f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:06.651362 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb338b12-0c45-4f5c-888a-14b95c177efc None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-783457a8-e592-4f80-8a21-95b973cc82f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:06.657594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:06.658548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:06.659347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:06.659347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:06.659347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:06.659347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:06.659347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:06.659347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:06.659347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:06.659347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:06.659347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:06.659347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:06.659347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:17:06.659347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:06.659347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 216/442] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:17:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:06.660026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783457a8-e592-4f80-8a21-95b973cc82f9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:06.692782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d92c4-73b5-43e0-bc62-c119b211ab7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:06.693612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d92c4-73b5-43e0-bc62-c119b211ab7a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:06.693940 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d92c4-73b5-43e0-bc62-c119b211ab7a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:06.694224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d92c4-73b5-43e0-bc62-c119b211ab7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:06.707970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d92c4-73b5-43e0-bc62-c119b211ab7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-e81d92c4-73b5-43e0-bc62-c119b211ab7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:06.717687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:06.718612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:06.719407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:06.719407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:06.719407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:06.719407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:06.719407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:06.719407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:06.719407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:06.719407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:06.719407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:06.719407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:06.719407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:06.719407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:06.721680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 227/443] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:17:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:06.722637 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d92c4-73b5-43e0-bc62-c119b211ab7a None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:06.751577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:06.782193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783457a8-e592-4f80-8a21-95b973cc82f9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:06.818506 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db4dfc-dd20-4c3f-b35b-2211c62c5270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:06.819399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db4dfc-dd20-4c3f-b35b-2211c62c5270 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:06.819665 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db4dfc-dd20-4c3f-b35b-2211c62c5270 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:06.819944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db4dfc-dd20-4c3f-b35b-2211c62c5270 None admin] PATH_INFO: `/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:06.821154 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db4dfc-dd20-4c3f-b35b-2211c62c5270 None admin] RBAC: Authorizing `identity:get_group(group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:06.822670 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db4dfc-dd20-4c3f-b35b-2211c62c5270 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:06.824472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 217/444] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Jul 19 15:17:06 2022] GET /identity/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:06.825294 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db4dfc-dd20-4c3f-b35b-2211c62c5270 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:06.827862 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:06.837121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d92c4-73b5-43e0-bc62-c119b211ab7a None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:06.870682 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616edb1-54b9-4fe1-9b30-7082ee9bebdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:06.871343 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616edb1-54b9-4fe1-9b30-7082ee9bebdd None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:06.871622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616edb1-54b9-4fe1-9b30-7082ee9bebdd None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:06.871904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616edb1-54b9-4fe1-9b30-7082ee9bebdd None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:06.872647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616edb1-54b9-4fe1-9b30-7082ee9bebdd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:06.872959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616edb1-54b9-4fe1-9b30-7082ee9bebdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:06.874358 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616edb1-54b9-4fe1-9b30-7082ee9bebdd None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:06.882120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 228/445] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:06.882886 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616edb1-54b9-4fe1-9b30-7082ee9bebdd None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:06.931204 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db4dfc-dd20-4c3f-b35b-2211c62c5270 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:06.963613 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:06.964224 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2e48a-ea37-4d95-8aad-f31167e7ce2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:06.964978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2e48a-ea37-4d95-8aad-f31167e7ce2f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:06.965325 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2e48a-ea37-4d95-8aad-f31167e7ce2f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:06.965644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2e48a-ea37-4d95-8aad-f31167e7ce2f None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:06.966425 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2e48a-ea37-4d95-8aad-f31167e7ce2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:06.966838 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2e48a-ea37-4d95-8aad-f31167e7ce2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:06.968106 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2e48a-ea37-4d95-8aad-f31167e7ce2f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:06.979199 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 218/446] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:06.980371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2e48a-ea37-4d95-8aad-f31167e7ce2f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:06.987307 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616edb1-54b9-4fe1-9b30-7082ee9bebdd None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:07.020878 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a541e-133d-40fd-a0b2-b9e61167df4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:07.021549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a541e-133d-40fd-a0b2-b9e61167df4a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:07.021829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a541e-133d-40fd-a0b2-b9e61167df4a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:07.022099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a541e-133d-40fd-a0b2-b9e61167df4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:07.022929 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a541e-133d-40fd-a0b2-b9e61167df4a None admin] RBAC: Adding query filter params (group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445, role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f, scope.project.id=2bda471206fe4fbb88370df9ab3bdfac) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:07.023248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a541e-133d-40fd-a0b2-b9e61167df4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:07.024688 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a541e-133d-40fd-a0b2-b9e61167df4a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:07.030636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 229/447] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Jul 19 15:17:06 2022] GET /identity/v3/role_assignments?group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445&scope.project.id=2bda471206fe4fbb88370df9ab3bdfac&role.id=d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:07.031401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a541e-133d-40fd-a0b2-b9e61167df4a None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:07.048632 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:07.095334 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2e48a-ea37-4d95-8aad-f31167e7ce2f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:07.138497 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1176cbf-cdec-45bd-a443-9f12a92f3d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:07.139173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1176cbf-cdec-45bd-a443-9f12a92f3d7e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:07.139457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1176cbf-cdec-45bd-a443-9f12a92f3d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:07.139734 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1176cbf-cdec-45bd-a443-9f12a92f3d7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:07.151315 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a541e-133d-40fd-a0b2-b9e61167df4a None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:07.190701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e3a79-6c61-43dd-b1f3-730b6a05d4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:07.191356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e3a79-6c61-43dd-b1f3-730b6a05d4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:07.191637 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e3a79-6c61-43dd-b1f3-730b6a05d4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:07.191975 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e3a79-6c61-43dd-b1f3-730b6a05d4d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:07.210546 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:07.223970 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1176cbf-cdec-45bd-a443-9f12a92f3d7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:07.226339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1176cbf-cdec-45bd-a443-9f12a92f3d7e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:07.275125 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e3a79-6c61-43dd-b1f3-730b6a05d4d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:07.276048 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e3a79-6c61-43dd-b1f3-730b6a05d4d2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:07.306984 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:07.354500 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 219/448] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:17:07.360247 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1176cbf-cdec-45bd-a443-9f12a92f3d7e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:07.404255 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 230/449] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:17:07.410184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e3a79-6c61-43dd-b1f3-730b6a05d4d2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:07.456633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:07.464967 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1176cbf-cdec-45bd-a443-9f12a92f3d7e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:07.499739 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16051224-0db8-44cd-9fa8-832fa24c966d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:07.500605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16051224-0db8-44cd-9fa8-832fa24c966d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:07.500890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16051224-0db8-44cd-9fa8-832fa24c966d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:07.501268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16051224-0db8-44cd-9fa8-832fa24c966d None admin] PATH_INFO: `/v3/projects/2bda471206fe4fbb88370df9ab3bdfac` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:07.502426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16051224-0db8-44cd-9fa8-832fa24c966d None admin] RBAC: Authorizing `identity:get_project(project_id=2bda471206fe4fbb88370df9ab3bdfac)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:07.503862 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16051224-0db8-44cd-9fa8-832fa24c966d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:07.505911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 220/450] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:17:07 2022] GET /identity/v3/projects/2bda471206fe4fbb88370df9ab3bdfac => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:07.509109 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e3a79-6c61-43dd-b1f3-730b6a05d4d2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:07.511287 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16051224-0db8-44cd-9fa8-832fa24c966d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:07.541887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:07.544435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6b353-a57e-40aa-9c24-12171c7cbda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:07.545332 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6b353-a57e-40aa-9c24-12171c7cbda6 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:07.545636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6b353-a57e-40aa-9c24-12171c7cbda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:07.545893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6b353-a57e-40aa-9c24-12171c7cbda6 None admin] PATH_INFO: `/v3/projects/5bb6648f966d4104a0136e3bb203cb76` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:07.547088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6b353-a57e-40aa-9c24-12171c7cbda6 None admin] RBAC: Authorizing `identity:get_project(project_id=5bb6648f966d4104a0136e3bb203cb76)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:07.548543 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6b353-a57e-40aa-9c24-12171c7cbda6 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:07.550283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 231/451] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:17:07 2022] GET /identity/v3/projects/5bb6648f966d4104a0136e3bb203cb76 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:07.556237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6b353-a57e-40aa-9c24-12171c7cbda6 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:07.610535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16051224-0db8-44cd-9fa8-832fa24c966d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:07.643874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190bbcf-42cf-4f3f-87c7-477847ba5dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:07.644731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190bbcf-42cf-4f3f-87c7-477847ba5dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:07.645038 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190bbcf-42cf-4f3f-87c7-477847ba5dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:07.645510 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190bbcf-42cf-4f3f-87c7-477847ba5dd3 None admin] PATH_INFO: `/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:07.646693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190bbcf-42cf-4f3f-87c7-477847ba5dd3 None admin] RBAC: Authorizing `identity:get_group(group_id=0bdc6ca90dda4a3ba38f9b41f6ee1445)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:07.648123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190bbcf-42cf-4f3f-87c7-477847ba5dd3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:07.649769 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 221/452] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Jul 19 15:17:07 2022] GET /identity/v3/groups/0bdc6ca90dda4a3ba38f9b41f6ee1445 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:07.655720 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6b353-a57e-40aa-9c24-12171c7cbda6 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:07.655848 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190bbcf-42cf-4f3f-87c7-477847ba5dd3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:07.670242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:07.690959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a44e56-acd5-4e2a-aec6-a7bb7f468041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:07.691819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a44e56-acd5-4e2a-aec6-a7bb7f468041 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:07.692211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a44e56-acd5-4e2a-aec6-a7bb7f468041 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:07.692657 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a44e56-acd5-4e2a-aec6-a7bb7f468041 None admin] PATH_INFO: `/v3/groups/2c7e2c2a6d83453bb6c63f3f7854fa03` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:07.693893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a44e56-acd5-4e2a-aec6-a7bb7f468041 None admin] RBAC: Authorizing `identity:get_group(group_id=2c7e2c2a6d83453bb6c63f3f7854fa03)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:07.695333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a44e56-acd5-4e2a-aec6-a7bb7f468041 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:07.696951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 232/453] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Jul 19 15:17:07 2022] GET /identity/v3/groups/2c7e2c2a6d83453bb6c63f3f7854fa03 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:07.702622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a44e56-acd5-4e2a-aec6-a7bb7f468041 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:07.759722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:07.762914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190bbcf-42cf-4f3f-87c7-477847ba5dd3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:07.797510 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec69f6-e5f7-4ed3-9b40-4ed4062fe089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:07.798154 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec69f6-e5f7-4ed3-9b40-4ed4062fe089 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:07.798436 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec69f6-e5f7-4ed3-9b40-4ed4062fe089 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:07.798708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec69f6-e5f7-4ed3-9b40-4ed4062fe089 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:07.799536 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec69f6-e5f7-4ed3-9b40-4ed4062fe089 None admin] RBAC: Adding query filter params (group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445, role.id=44bd561cc5594ffd9bcc27ef6dad0fc5, scope.project.id=2bda471206fe4fbb88370df9ab3bdfac) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:07.799846 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec69f6-e5f7-4ed3-9b40-4ed4062fe089 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:07.801310 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec69f6-e5f7-4ed3-9b40-4ed4062fe089 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:07.802502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a44e56-acd5-4e2a-aec6-a7bb7f468041 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:07.807162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 222/454] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Jul 19 15:17:07 2022] GET /identity/v3/role_assignments?group.id=0bdc6ca90dda4a3ba38f9b41f6ee1445&scope.project.id=2bda471206fe4fbb88370df9ab3bdfac&role.id=44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:07.842262 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b50ebe-c5fa-44f5-9d34-958da7f6c95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:07.842930 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b50ebe-c5fa-44f5-9d34-958da7f6c95d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:07.843221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b50ebe-c5fa-44f5-9d34-958da7f6c95d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:07.843496 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b50ebe-c5fa-44f5-9d34-958da7f6c95d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:07.844346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b50ebe-c5fa-44f5-9d34-958da7f6c95d None admin] RBAC: Adding query filter params (group.id=2c7e2c2a6d83453bb6c63f3f7854fa03, role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=5bb6648f966d4104a0136e3bb203cb76) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:07.844654 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b50ebe-c5fa-44f5-9d34-958da7f6c95d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:07.846146 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b50ebe-c5fa-44f5-9d34-958da7f6c95d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:07.865265 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 233/455] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Jul 19 15:17:07 2022] GET /identity/v3/role_assignments?group.id=2c7e2c2a6d83453bb6c63f3f7854fa03&scope.project.id=5bb6648f966d4104a0136e3bb203cb76&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:10.389262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5eac5f-6d26-45be-a872-6967965e93ff None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:10.389771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5eac5f-6d26-45be-a872-6967965e93ff None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:10.389845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5eac5f-6d26-45be-a872-6967965e93ff None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:10.391421 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 223/456] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:10.396866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c24310-166c-4fff-8c8d-725df44bf3b1 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:10.397321 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c24310-166c-4fff-8c8d-725df44bf3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:10.397501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c24310-166c-4fff-8c8d-725df44bf3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:10.421258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-d8c24310-166c-4fff-8c8d-725df44bf3b1 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:10.422448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d8c24310-166c-4fff-8c8d-725df44bf3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:10.483066 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:10.510931 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 234/457] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:17:10.535360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5eac5f-6d26-45be-a872-6967965e93ff None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:10.640823 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171c40a-9951-45c0-940f-fcef5cc9e324 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:10.641181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171c40a-9951-45c0-940f-fcef5cc9e324 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:10.641400 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171c40a-9951-45c0-940f-fcef5cc9e324 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:10.643009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 235/458] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:10.648534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a25ae-7eb3-42a9-a9d7-5b22ed414f3c None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:10.648825 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a25ae-7eb3-42a9-a9d7-5b22ed414f3c None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:10.649108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a25ae-7eb3-42a9-a9d7-5b22ed414f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:10.670266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-b26a25ae-7eb3-42a9-a9d7-5b22ed414f3c None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:10.671472 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b26a25ae-7eb3-42a9-a9d7-5b22ed414f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:10.677828 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5eac5f-6d26-45be-a872-6967965e93ff None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:10.714627 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759bbc8-eb9e-4265-b812-51fe2c0fbc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:10.715420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759bbc8-eb9e-4265-b812-51fe2c0fbc6f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:10.715698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759bbc8-eb9e-4265-b812-51fe2c0fbc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:10.715966 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759bbc8-eb9e-4265-b812-51fe2c0fbc6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:10.726636 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759bbc8-eb9e-4265-b812-51fe2c0fbc6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:10.728093 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759bbc8-eb9e-4265-b812-51fe2c0fbc6f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:10.741601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 236/459] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:17:10.743075 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c9974-bd3e-4e61-bb8c-48fb68dc1729 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:10.743371 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c9974-bd3e-4e61-bb8c-48fb68dc1729 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:10.743651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c9974-bd3e-4e61-bb8c-48fb68dc1729 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:10.745237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 237/460] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:10.746605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c759bbc8-eb9e-4265-b812-51fe2c0fbc6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:10.746605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:10.746605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:10.746605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:10.746605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:10.746605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:10.746605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:10.746605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:10.746605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:10.746605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:10.746605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:10.746605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:10.746605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:10.746605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:10.746605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:10.747438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:10.748712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:10.748712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:10.748712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:10.748712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:10.748712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 224/461] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:10.753924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759bbc8-eb9e-4265-b812-51fe2c0fbc6f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:10.754223 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8388cce-dcf9-4568-8d1a-00a6618a80c8 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:10.755202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8388cce-dcf9-4568-8d1a-00a6618a80c8 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:10.755646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8388cce-dcf9-4568-8d1a-00a6618a80c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:10.773232 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:10.784306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-e8388cce-dcf9-4568-8d1a-00a6618a80c8 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:10.786017 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e8388cce-dcf9-4568-8d1a-00a6618a80c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:10.862249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759bbc8-eb9e-4265-b812-51fe2c0fbc6f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:10.866455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 238/462] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:10.872163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8388cce-dcf9-4568-8d1a-00a6618a80c8 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:10.911859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:10.928302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951e7e1-181a-4440-aa4a-269f17476fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:10.928976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951e7e1-181a-4440-aa4a-269f17476fec None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:10.929290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951e7e1-181a-4440-aa4a-269f17476fec None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:10.929565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951e7e1-181a-4440-aa4a-269f17476fec None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:10.930298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951e7e1-181a-4440-aa4a-269f17476fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:10.930604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951e7e1-181a-4440-aa4a-269f17476fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:10.931970 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951e7e1-181a-4440-aa4a-269f17476fec None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:10.945275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 225/463] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:10.945970 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951e7e1-181a-4440-aa4a-269f17476fec None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:10.994844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8388cce-dcf9-4568-8d1a-00a6618a80c8 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:11.032602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca0691-0575-4096-b0d6-9b0ba850ac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:11.033773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca0691-0575-4096-b0d6-9b0ba850ac48 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:11.034276 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca0691-0575-4096-b0d6-9b0ba850ac48 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:11.034749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca0691-0575-4096-b0d6-9b0ba850ac48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:11.041502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:11.046226 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca0691-0575-4096-b0d6-9b0ba850ac48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:11.047992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca0691-0575-4096-b0d6-9b0ba850ac48 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:11.055520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951e7e1-181a-4440-aa4a-269f17476fec None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:11.068423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-b0ca0691-0575-4096-b0d6-9b0ba850ac48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:11.068423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:11.068423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:11.068423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:11.068423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:11.068423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:11.068423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:11.068423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:11.068423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:11.068423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:11.068423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:11.068423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:11.068423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:11.068423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:11.068423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:11.069346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:11.070256 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:11.070256 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:11.070256 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:11.070256 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:11.077335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 239/464] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:11.078409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca0691-0575-4096-b0d6-9b0ba850ac48 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:11.093250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffeb1d-616f-4bd6-80b7-8ac01d979c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:11.094163 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffeb1d-616f-4bd6-80b7-8ac01d979c4e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:11.094541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffeb1d-616f-4bd6-80b7-8ac01d979c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:11.094897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffeb1d-616f-4bd6-80b7-8ac01d979c4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:11.106254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffeb1d-616f-4bd6-80b7-8ac01d979c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:11.107850 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffeb1d-616f-4bd6-80b7-8ac01d979c4e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:11.126639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ecffeb1d-616f-4bd6-80b7-8ac01d979c4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:11.126639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:11.126639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:11.126639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:11.126639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:11.126639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:11.126639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:11.126639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:11.126639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:11.126639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:11.126639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:11.126639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:11.126639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:11.126639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:11.126639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:11.127513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:11.128364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:11.128364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:11.128364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:11.128364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:11.132919 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 226/465] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:11.133885 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffeb1d-616f-4bd6-80b7-8ac01d979c4e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:11.178247 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:11.192482 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca0691-0575-4096-b0d6-9b0ba850ac48 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:11.228917 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fe6f8-cc40-4d3a-a59c-b7ea0fd188ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:11.229886 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fe6f8-cc40-4d3a-a59c-b7ea0fd188ab None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:11.230396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fe6f8-cc40-4d3a-a59c-b7ea0fd188ab None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:11.230861 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fe6f8-cc40-4d3a-a59c-b7ea0fd188ab None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:11.231877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fe6f8-cc40-4d3a-a59c-b7ea0fd188ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:11.233603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fe6f8-cc40-4d3a-a59c-b7ea0fd188ab None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:11.241497 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffeb1d-616f-4bd6-80b7-8ac01d979c4e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:11.266515 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 240/466] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:17:11 2022] POST /identity/v3/users => generated 246 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:17:11.267801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fe6f8-cc40-4d3a-a59c-b7ea0fd188ab None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:11.272036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:11.301993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4cea9-f190-4791-8413-e03b384decb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:11.302469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4cea9-f190-4791-8413-e03b384decb2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:11.302827 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4cea9-f190-4791-8413-e03b384decb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:11.303092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4cea9-f190-4791-8413-e03b384decb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:11.303860 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4cea9-f190-4791-8413-e03b384decb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:11.304177 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4cea9-f190-4791-8413-e03b384decb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:11.306303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4cea9-f190-4791-8413-e03b384decb2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:11.319460 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 227/467] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:11.327352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4cea9-f190-4791-8413-e03b384decb2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:11.400206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fe6f8-cc40-4d3a-a59c-b7ea0fd188ab None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:11.446894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0cf93-64cc-4283-9f9d-4ef26b0e22b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:11.447655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0cf93-64cc-4283-9f9d-4ef26b0e22b4 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:11.447942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0cf93-64cc-4283-9f9d-4ef26b0e22b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:11.448217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0cf93-64cc-4283-9f9d-4ef26b0e22b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:11.448951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0cf93-64cc-4283-9f9d-4ef26b0e22b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:11.449303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0cf93-64cc-4283-9f9d-4ef26b0e22b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:11.450688 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0cf93-64cc-4283-9f9d-4ef26b0e22b4 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:11.451003 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4cea9-f190-4791-8413-e03b384decb2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:11.471574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 241/468] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:11.473910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c88a2-8399-40d6-9971-8e68343c3761 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:11.474229 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c88a2-8399-40d6-9971-8e68343c3761 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:11.474502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c88a2-8399-40d6-9971-8e68343c3761 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:11.475950 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 242/469] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:11.480532 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c88a2-8399-40d6-9971-8e68343c3761 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:11.484640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:11.501715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6e7b6-10fe-4a53-9895-fe66903b1d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:11.502418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6e7b6-10fe-4a53-9895-fe66903b1d10 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:11.502698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6e7b6-10fe-4a53-9895-fe66903b1d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:11.503008 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6e7b6-10fe-4a53-9895-fe66903b1d10 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:11.503844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6e7b6-10fe-4a53-9895-fe66903b1d10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:11.505529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6e7b6-10fe-4a53-9895-fe66903b1d10 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:11.528093 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 228/470] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:17:11 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:17:11.533443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead019b6-508d-43dd-8bec-f054d23b12cd None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:11.533775 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead019b6-508d-43dd-8bec-f054d23b12cd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:11.534055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead019b6-508d-43dd-8bec-f054d23b12cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:11.569293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ead019b6-508d-43dd-8bec-f054d23b12cd None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:11.571851 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ead019b6-508d-43dd-8bec-f054d23b12cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:11.622928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c88a2-8399-40d6-9971-8e68343c3761 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:11.635418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:11.678362 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 229/471] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:11.703782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2624d-e8d8-4328-85bb-8f628ab10379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:11.704861 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2624d-e8d8-4328-85bb-8f628ab10379 None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:11.705394 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2624d-e8d8-4328-85bb-8f628ab10379 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:11.705936 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2624d-e8d8-4328-85bb-8f628ab10379 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:11.706860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2624d-e8d8-4328-85bb-8f628ab10379 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:11.706978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead019b6-508d-43dd-8bec-f054d23b12cd None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:11.709393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2624d-e8d8-4328-85bb-8f628ab10379 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:11.738110 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 243/472] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:17:11 2022] POST /identity/v3/users => generated 248 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:17:11.835693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead019b6-508d-43dd-8bec-f054d23b12cd None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:11.874298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ea6b1-84b7-434a-a01c-c4eff6a3dcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:11.875165 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ea6b1-84b7-434a-a01c-c4eff6a3dcf3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:11.875451 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ea6b1-84b7-434a-a01c-c4eff6a3dcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:11.875723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ea6b1-84b7-434a-a01c-c4eff6a3dcf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:11.888170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ea6b1-84b7-434a-a01c-c4eff6a3dcf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:11.889677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ea6b1-84b7-434a-a01c-c4eff6a3dcf3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:11.909547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-626ea6b1-84b7-434a-a01c-c4eff6a3dcf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:11.909547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:11.909547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:11.909547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:11.909547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:11.909547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:11.909547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:11.909547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:11.909547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:11.909547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:11.909547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:11.909547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:11.909547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:11.909547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:11.909547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:11.910505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:11.911345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:11.911345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:11.911345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:11.911345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:11.911345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 230/473] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:11.917049 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2624d-e8d8-4328-85bb-8f628ab10379 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:11.990046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:12.035184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2624d-e8d8-4328-85bb-8f628ab10379 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:12.048445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93548c2-54c9-4bb5-b1db-6c40368163d9 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:12.048740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93548c2-54c9-4bb5-b1db-6c40368163d9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:12.049025 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93548c2-54c9-4bb5-b1db-6c40368163d9 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:12.050660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 231/474] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:17:12.056497 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93046680-5862-4cf8-9080-7314447e20a3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:12.057044 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93046680-5862-4cf8-9080-7314447e20a3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:12.057522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93046680-5862-4cf8-9080-7314447e20a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:12.075372 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-93046680-5862-4cf8-9080-7314447e20a3 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:12.076931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-93046680-5862-4cf8-9080-7314447e20a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:12.090025 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52b4be-3efc-48d1-8c86-bdaad99cf9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:12.090786 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52b4be-3efc-48d1-8c86-bdaad99cf9aa None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:12.091073 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52b4be-3efc-48d1-8c86-bdaad99cf9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:12.091354 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52b4be-3efc-48d1-8c86-bdaad99cf9aa None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:12.092137 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52b4be-3efc-48d1-8c86-bdaad99cf9aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:12.092429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52b4be-3efc-48d1-8c86-bdaad99cf9aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:12.093996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52b4be-3efc-48d1-8c86-bdaad99cf9aa None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:12.103323 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:12.116185 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 244/475] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:12.126226 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52b4be-3efc-48d1-8c86-bdaad99cf9aa None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:12.172423 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 232/476] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:17:12.198936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93046680-5862-4cf8-9080-7314447e20a3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:12.250424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52b4be-3efc-48d1-8c86-bdaad99cf9aa None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:12.275856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:12.292705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea179e24-aa1f-4ba2-99f1-38eff64ff3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:12.293406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea179e24-aa1f-4ba2-99f1-38eff64ff3b6 None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:12.293686 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea179e24-aa1f-4ba2-99f1-38eff64ff3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:12.293957 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea179e24-aa1f-4ba2-99f1-38eff64ff3b6 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:12.294731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea179e24-aa1f-4ba2-99f1-38eff64ff3b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:12.296680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea179e24-aa1f-4ba2-99f1-38eff64ff3b6 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:12.319883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 245/477] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:17:12 2022] POST /identity/v3/users => generated 249 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:17:12.420309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93046680-5862-4cf8-9080-7314447e20a3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:12.481946 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92967e02-67b6-481c-a5de-a31b1f66b4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:12.482703 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92967e02-67b6-481c-a5de-a31b1f66b4ca None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:12.483017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92967e02-67b6-481c-a5de-a31b1f66b4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:12.483331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92967e02-67b6-481c-a5de-a31b1f66b4ca None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:12.484147 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92967e02-67b6-481c-a5de-a31b1f66b4ca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:12.485682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92967e02-67b6-481c-a5de-a31b1f66b4ca None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:12.500443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.sql.core [None req-92967e02-67b6-481c-a5de-a31b1f66b4ca None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 19 15:17:12.500443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 19 15:17:12.500443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [parameters: {'id': '85b0e4d0699f442cbdbeeb0962b06ec6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 19 15:17:12.500443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80270) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 19 15:17:12.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-92967e02-67b6-481c-a5de-a31b1f66b4ca None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 19 15:17:12.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:12.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 15:17:12.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 15:17:12.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 15:17:12.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 15:17:12.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 15:17:12.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 15:17:12.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 15:17:12.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application conn.query(q) Jul 19 15:17:12.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 15:17:12.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 15:17:12.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 15:17:12.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result.read() Jul 19 15:17:12.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return ref.to_dict() Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 15:17:12.519454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application next(self.gen) Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application yield resource Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application next(self.gen) Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.session.rollback() Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.force_reraise() Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise self.value Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application session.commit() Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 19 15:17:12.520903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._prepare_impl() Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.session.flush() Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._flush(objects) Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application compat.raise_( Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application flush_context.execute() Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rec.execute(self) Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 19 15:17:12.523111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 15:17:12.525214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application conn.query(q) Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result.read() Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 15:17:12.527409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [parameters: {'id': '85b0e4d0699f442cbdbeeb0962b06ec6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:12.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 19 15:17:12.534383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:12.538085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 233/478] 10.222.0.42 () {60 vars in 1131 bytes} [Tue Jul 19 15:17:12 2022] POST /identity/v3/roles => generated 144 bytes in 323 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:12.538223 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea179e24-aa1f-4ba2-99f1-38eff64ff3b6 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:12.650703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea179e24-aa1f-4ba2-99f1-38eff64ff3b6 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:12.687769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264874a-2408-4430-9c17-cdc7ff33eb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:12.688895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264874a-2408-4430-9c17-cdc7ff33eb96 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:12.689444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264874a-2408-4430-9c17-cdc7ff33eb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:12.689987 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264874a-2408-4430-9c17-cdc7ff33eb96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:12.701873 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264874a-2408-4430-9c17-cdc7ff33eb96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-1264874a-2408-4430-9c17-cdc7ff33eb96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:12.710833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:12.711769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:12.712620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:12.712620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:12.712620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:12.712620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:12.712620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:12.712620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:12.712620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:12.712620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:12.712620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:12.712620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:12.712620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 15:17:12.712620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:12.715145 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 246/479] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:17:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:12.722241 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92967e02-67b6-481c-a5de-a31b1f66b4ca None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:12.761727 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:12.859760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92967e02-67b6-481c-a5de-a31b1f66b4ca None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:12.905894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7333a44-878d-4977-bf1b-a33bd97ce83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:12.906622 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7333a44-878d-4977-bf1b-a33bd97ce83b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:12.906916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7333a44-878d-4977-bf1b-a33bd97ce83b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:12.907233 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7333a44-878d-4977-bf1b-a33bd97ce83b None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:12.907995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7333a44-878d-4977-bf1b-a33bd97ce83b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:12.908311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7333a44-878d-4977-bf1b-a33bd97ce83b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:12.909739 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7333a44-878d-4977-bf1b-a33bd97ce83b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:12.919296 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 234/480] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:13.791299 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb09ee-0c77-4223-8d6e-0da73fbbcd67 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:13.792351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb09ee-0c77-4223-8d6e-0da73fbbcd67 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:13.792839 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb09ee-0c77-4223-8d6e-0da73fbbcd67 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:13.794695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 247/481] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:13.799971 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21f727-0f56-4963-96b9-30e796fa8616 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:13.800769 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21f727-0f56-4963-96b9-30e796fa8616 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:13.801344 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21f727-0f56-4963-96b9-30e796fa8616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:13.819224 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-da21f727-0f56-4963-96b9-30e796fa8616 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:13.820742 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-da21f727-0f56-4963-96b9-30e796fa8616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:13.891254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 235/482] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:13.916460 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb09ee-0c77-4223-8d6e-0da73fbbcd67 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:13.966165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:14.042489 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb09ee-0c77-4223-8d6e-0da73fbbcd67 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:14.090827 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88382b50-1345-4eb3-bb1a-9bdba68f6e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:14.091978 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88382b50-1345-4eb3-bb1a-9bdba68f6e04 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:14.092463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88382b50-1345-4eb3-bb1a-9bdba68f6e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:14.092956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88382b50-1345-4eb3-bb1a-9bdba68f6e04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:14.113784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88382b50-1345-4eb3-bb1a-9bdba68f6e04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-88382b50-1345-4eb3-bb1a-9bdba68f6e04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:14.122273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:14.123286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:14.124086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:14.124086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:14.124086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:14.124086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:14.124086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:14.124086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:14.124086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:14.124086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:14.124086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:14.124086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:14.124086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:14.124086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:14.126583 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 248/483] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:14.133248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21f727-0f56-4963-96b9-30e796fa8616 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:14.153503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:14.279927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21f727-0f56-4963-96b9-30e796fa8616 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:14.294903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de107c1-7d69-4e3f-90c6-c8e7730591e5 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:14.295501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de107c1-7d69-4e3f-90c6-c8e7730591e5 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:14.295995 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de107c1-7d69-4e3f-90c6-c8e7730591e5 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:14.297876 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 249/484] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:17:14.303266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ac705-7a72-495e-9537-fe5f435287df None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:14.303792 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ac705-7a72-495e-9537-fe5f435287df None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:14.304268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ac705-7a72-495e-9537-fe5f435287df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:14.321927 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-c71ac705-7a72-495e-9537-fe5f435287df None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:14.323496 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c71ac705-7a72-495e-9537-fe5f435287df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:14.337942 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449d56b-8d50-4d34-a9cd-cab3f3725da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:14.338633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449d56b-8d50-4d34-a9cd-cab3f3725da5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:14.338922 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449d56b-8d50-4d34-a9cd-cab3f3725da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:14.339200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449d56b-8d50-4d34-a9cd-cab3f3725da5 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:14.340080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449d56b-8d50-4d34-a9cd-cab3f3725da5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:14.340397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449d56b-8d50-4d34-a9cd-cab3f3725da5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:14.341915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449d56b-8d50-4d34-a9cd-cab3f3725da5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:14.357785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 236/485] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:14.363755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449d56b-8d50-4d34-a9cd-cab3f3725da5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:14.419873 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 250/486] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:17:14.421990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bba0c-0cd0-45f9-b7f6-fa71810f51b6 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:14.422591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bba0c-0cd0-45f9-b7f6-fa71810f51b6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:14.423067 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bba0c-0cd0-45f9-b7f6-fa71810f51b6 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:14.424572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 251/487] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:14.430493 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7a0c4-9eec-4ebc-be7a-9e4224249b3b None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:14.430647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7a0c4-9eec-4ebc-be7a-9e4224249b3b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:14.430924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7a0c4-9eec-4ebc-be7a-9e4224249b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:14.448762 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-22f7a0c4-9eec-4ebc-be7a-9e4224249b3b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:14.449864 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-22f7a0c4-9eec-4ebc-be7a-9e4224249b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:14.470730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:14.510817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:14.536837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 252/488] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:17:14.538022 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7a0c4-9eec-4ebc-be7a-9e4224249b3b None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:14.553897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449d56b-8d50-4d34-a9cd-cab3f3725da5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:14.590088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788580c7-798f-4f99-8b82-2bad6d306617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:14.590894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788580c7-798f-4f99-8b82-2bad6d306617 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:14.591180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788580c7-798f-4f99-8b82-2bad6d306617 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:14.591458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788580c7-798f-4f99-8b82-2bad6d306617 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:14.602771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788580c7-798f-4f99-8b82-2bad6d306617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:14.604232 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788580c7-798f-4f99-8b82-2bad6d306617 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:14.622400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-788580c7-798f-4f99-8b82-2bad6d306617 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:14.622400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:14.622400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:14.622400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:14.622400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:14.622400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:14.622400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:14.622400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:14.622400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:14.622400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:14.622400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:14.622400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:14.622400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:14.622400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:14.622400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:14.623275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:14.624137 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:14.624137 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:14.624137 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:14.624137 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:14.628225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 237/489] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:14.629662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788580c7-798f-4f99-8b82-2bad6d306617 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:14.668353 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7a0c4-9eec-4ebc-be7a-9e4224249b3b None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:14.707756 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a056f97-dc92-4956-825d-d59e2c553cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:14.708630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a056f97-dc92-4956-825d-d59e2c553cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:14.708916 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a056f97-dc92-4956-825d-d59e2c553cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:14.709215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a056f97-dc92-4956-825d-d59e2c553cf2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:14.722428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a056f97-dc92-4956-825d-d59e2c553cf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-4a056f97-dc92-4956-825d-d59e2c553cf2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:14.731001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:14.731918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:14.732712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:14.732712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:14.732712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:14.732712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:14.732712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:14.732712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:14.732712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:14.732712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:14.732712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:14.732712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:14.732712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:14.732712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:14.737182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 253/490] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:14.738043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a056f97-dc92-4956-825d-d59e2c553cf2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:14.739306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788580c7-798f-4f99-8b82-2bad6d306617 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:14.759492 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:14.766898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:14.778149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb43581-e5db-431d-980b-0a8919eb8d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:14.779389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb43581-e5db-431d-980b-0a8919eb8d1d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:14.779687 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb43581-e5db-431d-980b-0a8919eb8d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:14.779970 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb43581-e5db-431d-980b-0a8919eb8d1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:14.794443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb43581-e5db-431d-980b-0a8919eb8d1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-4fb43581-e5db-431d-980b-0a8919eb8d1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:14.805683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:14.807227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:14.808218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:14.808218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:14.808218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:14.808218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:14.808218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:14.808218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:14.808218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:14.808218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:14.808218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:14.808218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:14.808218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:14.808218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:14.808218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 238/491] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:14.808218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb43581-e5db-431d-980b-0a8919eb8d1d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:14.879504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a056f97-dc92-4956-825d-d59e2c553cf2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:14.916530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b03a4-d362-4cb4-b2f0-885119bf33ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:14.917230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b03a4-d362-4cb4-b2f0-885119bf33ac None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:14.917521 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b03a4-d362-4cb4-b2f0-885119bf33ac None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:14.918080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b03a4-d362-4cb4-b2f0-885119bf33ac None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:14.918537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b03a4-d362-4cb4-b2f0-885119bf33ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:14.918851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b03a4-d362-4cb4-b2f0-885119bf33ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:14.920259 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b03a4-d362-4cb4-b2f0-885119bf33ac None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:14.935124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 254/492] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:14.935953 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b03a4-d362-4cb4-b2f0-885119bf33ac None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:14.943709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb43581-e5db-431d-980b-0a8919eb8d1d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:14.991969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de8dd5-b5d6-465b-9bde-230abf8b3dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:14.993016 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de8dd5-b5d6-465b-9bde-230abf8b3dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:14.994085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de8dd5-b5d6-465b-9bde-230abf8b3dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:14.994573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de8dd5-b5d6-465b-9bde-230abf8b3dc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:14.995778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de8dd5-b5d6-465b-9bde-230abf8b3dc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:14.996500 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de8dd5-b5d6-465b-9bde-230abf8b3dc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:14.998804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de8dd5-b5d6-465b-9bde-230abf8b3dc3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:15.011251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 239/493] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:15.012507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de8dd5-b5d6-465b-9bde-230abf8b3dc3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:15.024891 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:15.054798 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b03a4-d362-4cb4-b2f0-885119bf33ac None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:15.075642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:15.103603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e66018-a4a4-406f-b67a-a8682ecb44d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:15.104654 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e66018-a4a4-406f-b67a-a8682ecb44d6 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:15.105162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e66018-a4a4-406f-b67a-a8682ecb44d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:15.106161 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e66018-a4a4-406f-b67a-a8682ecb44d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:15.107138 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e66018-a4a4-406f-b67a-a8682ecb44d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:15.108071 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e66018-a4a4-406f-b67a-a8682ecb44d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:15.110368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e66018-a4a4-406f-b67a-a8682ecb44d6 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:15.121806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 255/494] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:15.123293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e66018-a4a4-406f-b67a-a8682ecb44d6 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:15.138386 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de8dd5-b5d6-465b-9bde-230abf8b3dc3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:15.181818 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e78602-278a-41ed-8d60-fb501b6838ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:15.182624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e78602-278a-41ed-8d60-fb501b6838ce None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:15.182912 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e78602-278a-41ed-8d60-fb501b6838ce None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:15.183198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e78602-278a-41ed-8d60-fb501b6838ce None admin] PATH_INFO: `/v3/users/nova` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:15.192654 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e78602-278a-41ed-8d60-fb501b6838ce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:15.194162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e78602-278a-41ed-8d60-fb501b6838ce None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-b7e78602-278a-41ed-8d60-fb501b6838ce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:15.210145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:15.211075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:15.211979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:15.211979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:15.211979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:15.211979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:15.211979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:15.211979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:15.211979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:15.211979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:15.211979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:17:15.211979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:15.217883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 240/495] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Jul 19 15:17:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:15.217883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e78602-278a-41ed-8d60-fb501b6838ce None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:15.240797 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e66018-a4a4-406f-b67a-a8682ecb44d6 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:15.278856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c2d12-ad7f-44bd-9e86-8b7a7c4557b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:15.279659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c2d12-ad7f-44bd-9e86-8b7a7c4557b3 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:15.279947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c2d12-ad7f-44bd-9e86-8b7a7c4557b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:15.280219 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c2d12-ad7f-44bd-9e86-8b7a7c4557b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:15.288200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:15.293622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c2d12-ad7f-44bd-9e86-8b7a7c4557b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:15.295284 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c2d12-ad7f-44bd-9e86-8b7a7c4557b3 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:15.314384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-d39c2d12-ad7f-44bd-9e86-8b7a7c4557b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:15.314384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:15.314384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:15.314384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:15.314384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:15.314384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:15.314384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:15.314384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:15.314384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:15.314384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:15.314384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:15.314384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:15.314384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:15.314384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:15.314384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:15.315274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:15.316134 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:15.316134 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:15.316134 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:15.316134 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:15.316134 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 256/496] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:15.318159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1237248-7a87-4c32-b3cb-87f7ee0f7784 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:15.318451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1237248-7a87-4c32-b3cb-87f7ee0f7784 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:15.318733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1237248-7a87-4c32-b3cb-87f7ee0f7784 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:15.320152 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 257/497] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:15.320962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1237248-7a87-4c32-b3cb-87f7ee0f7784 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:15.343281 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e78602-278a-41ed-8d60-fb501b6838ce None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:15.353157 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:15.382743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3087c1-aa5b-46cd-9ac8-2b6d28129e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:15.383662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3087c1-aa5b-46cd-9ac8-2b6d28129e12 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:15.384085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3087c1-aa5b-46cd-9ac8-2b6d28129e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:15.384370 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3087c1-aa5b-46cd-9ac8-2b6d28129e12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:15.396344 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3087c1-aa5b-46cd-9ac8-2b6d28129e12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:15.398112 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3087c1-aa5b-46cd-9ac8-2b6d28129e12 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:15.416804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ac3087c1-aa5b-46cd-9ac8-2b6d28129e12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:15.416804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:15.416804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:15.416804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:15.416804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:15.416804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:15.416804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:15.416804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:15.416804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:15.416804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:15.416804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:15.416804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:15.416804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:15.416804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:15.416804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:15.417715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:15.418560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:15.418560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:15.418560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:15.418560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:15.418560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 241/498] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:15.419259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3087c1-aa5b-46cd-9ac8-2b6d28129e12 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:15.434154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1237248-7a87-4c32-b3cb-87f7ee0f7784 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:15.470156 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed99ba-2f4b-4917-9433-09f7704e93d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:15.470834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed99ba-2f4b-4917-9433-09f7704e93d3 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:15.471121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed99ba-2f4b-4917-9433-09f7704e93d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:15.471405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed99ba-2f4b-4917-9433-09f7704e93d3 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:15.472277 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed99ba-2f4b-4917-9433-09f7704e93d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:15.472617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed99ba-2f4b-4917-9433-09f7704e93d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:15.474162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed99ba-2f4b-4917-9433-09f7704e93d3 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:15.511657 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 258/499] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Jul 19 15:17:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:15.512941 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c4fcd-3040-436c-a982-60f3729f6e4e None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:15.513267 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c4fcd-3040-436c-a982-60f3729f6e4e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:15.513551 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c4fcd-3040-436c-a982-60f3729f6e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:15.531253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-676c4fcd-3040-436c-a982-60f3729f6e4e None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:15.532468 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-676c4fcd-3040-436c-a982-60f3729f6e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:15.533248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3087c1-aa5b-46cd-9ac8-2b6d28129e12 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:15.569663 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb8da2-e1d1-4fa1-9e75-5660b6724fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:15.570634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb8da2-e1d1-4fa1-9e75-5660b6724fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:15.571122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb8da2-e1d1-4fa1-9e75-5660b6724fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:15.571594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb8da2-e1d1-4fa1-9e75-5660b6724fe8 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:15.572623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb8da2-e1d1-4fa1-9e75-5660b6724fe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:15.573009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb8da2-e1d1-4fa1-9e75-5660b6724fe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:15.574552 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb8da2-e1d1-4fa1-9e75-5660b6724fe8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:15.576136 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:15.591861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 242/500] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:15.591861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb8da2-e1d1-4fa1-9e75-5660b6724fe8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:15.613243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 259/501] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:15.613243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c4fcd-3040-436c-a982-60f3729f6e4e None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:15.620798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:15.704608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb8da2-e1d1-4fa1-9e75-5660b6724fe8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:15.714972 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c4fcd-3040-436c-a982-60f3729f6e4e None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:15.741760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd6d6b-c0f8-47e5-83e8-1c2714f6dd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:15.742546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd6d6b-c0f8-47e5-83e8-1c2714f6dd17 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:15.742829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd6d6b-c0f8-47e5-83e8-1c2714f6dd17 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:15.743108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd6d6b-c0f8-47e5-83e8-1c2714f6dd17 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:15.743841 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd6d6b-c0f8-47e5-83e8-1c2714f6dd17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:15.744156 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd6d6b-c0f8-47e5-83e8-1c2714f6dd17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:15.745808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd6d6b-c0f8-47e5-83e8-1c2714f6dd17 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:15.750073 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f1219-0a28-4700-bb35-181a6bd2c943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:15.750725 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f1219-0a28-4700-bb35-181a6bd2c943 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:15.751011 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f1219-0a28-4700-bb35-181a6bd2c943 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:15.751311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f1219-0a28-4700-bb35-181a6bd2c943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:15.762390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 243/502] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:15.762390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd6d6b-c0f8-47e5-83e8-1c2714f6dd17 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:15.802409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:15.826068 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f1219-0a28-4700-bb35-181a6bd2c943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:15.827572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f1219-0a28-4700-bb35-181a6bd2c943 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:15.842032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:15.862313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd6d6b-c0f8-47e5-83e8-1c2714f6dd17 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:15.897345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f4adc-7ce4-4d33-b3a0-d55b730d0849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:15.898131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f4adc-7ce4-4d33-b3a0-d55b730d0849 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:15.898421 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f4adc-7ce4-4d33-b3a0-d55b730d0849 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:15.898703 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f4adc-7ce4-4d33-b3a0-d55b730d0849 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:15.906603 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f4adc-7ce4-4d33-b3a0-d55b730d0849 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:15.908064 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f4adc-7ce4-4d33-b3a0-d55b730d0849 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-b63f4adc-7ce4-4d33-b3a0-d55b730d0849 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:15.922035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:15.923001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:15.923897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:15.923897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:15.923897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:15.923897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:15.923897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:15.923897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:15.923897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:15.923897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:15.923897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 19 15:17:15.923897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:15.923897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 244/503] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:15.924494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f4adc-7ce4-4d33-b3a0-d55b730d0849 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:15.935441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 260/504] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:17:15.935441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f1219-0a28-4700-bb35-181a6bd2c943 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:16.012602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:16.024601 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f4adc-7ce4-4d33-b3a0-d55b730d0849 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:16.033759 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f1219-0a28-4700-bb35-181a6bd2c943 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:16.057573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2521dc6-6e52-4b90-89cd-5155afebc7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:16.058364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2521dc6-6e52-4b90-89cd-5155afebc7cb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:16.058645 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2521dc6-6e52-4b90-89cd-5155afebc7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:16.058922 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2521dc6-6e52-4b90-89cd-5155afebc7cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:16.066469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:16.066796 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284dcbaf-7697-4062-b7cc-c3825aab04af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:16.067581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dcbaf-7697-4062-b7cc-c3825aab04af None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:16.067903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dcbaf-7697-4062-b7cc-c3825aab04af None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:16.068119 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dcbaf-7697-4062-b7cc-c3825aab04af None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:16.070801 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2521dc6-6e52-4b90-89cd-5155afebc7cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:16.075533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284dcbaf-7697-4062-b7cc-c3825aab04af None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:16.077009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284dcbaf-7697-4062-b7cc-c3825aab04af None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c2521dc6-6e52-4b90-89cd-5155afebc7cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:16.078618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:16.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:16.080542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:16.080542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:16.080542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:16.080542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:16.080542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:16.080542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:16.080542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:16.080542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:16.080542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:16.080542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:16.080542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:16.080542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:16.080542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 245/505] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:16.081233 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26f562-845a-47ad-ae3f-fd946372f331 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:16.081690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26f562-845a-47ad-ae3f-fd946372f331 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:16.081690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26f562-845a-47ad-ae3f-fd946372f331 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:16.085180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 246/506] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:16.085180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26f562-845a-47ad-ae3f-fd946372f331 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-284dcbaf-7697-4062-b7cc-c3825aab04af None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:16.095653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:16.096552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:16.097459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:16.097459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:16.097459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:16.097459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:16.097459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:16.097459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:16.097459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:16.097459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:16.097459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 15:17:16.097459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:16.097459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 261/507] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:16.098722 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284dcbaf-7697-4062-b7cc-c3825aab04af None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:16.191921 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26f562-845a-47ad-ae3f-fd946372f331 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:16.203990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284dcbaf-7697-4062-b7cc-c3825aab04af None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:16.227875 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d73379-8162-45be-ae38-363ba7dac3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:16.228304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d73379-8162-45be-ae38-363ba7dac3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:16.228587 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d73379-8162-45be-ae38-363ba7dac3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:16.228860 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d73379-8162-45be-ae38-363ba7dac3e2 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:16.229737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d73379-8162-45be-ae38-363ba7dac3e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:16.230050 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d73379-8162-45be-ae38-363ba7dac3e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:16.231479 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d73379-8162-45be-ae38-363ba7dac3e2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:16.238350 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa33e6f-fefc-4f0f-bf53-33ec335fa055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:16.239138 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa33e6f-fefc-4f0f-bf53-33ec335fa055 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:16.239424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa33e6f-fefc-4f0f-bf53-33ec335fa055 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:16.239698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa33e6f-fefc-4f0f-bf53-33ec335fa055 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:16.246469 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:16.251059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa33e6f-fefc-4f0f-bf53-33ec335fa055 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:16.253040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa33e6f-fefc-4f0f-bf53-33ec335fa055 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:16.267458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 247/508] 10.222.0.42 () {56 vars in 1137 bytes} [Tue Jul 19 15:17:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:16.268290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d73379-8162-45be-ae38-363ba7dac3e2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:16.272524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-5fa33e6f-fefc-4f0f-bf53-33ec335fa055 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:16.272524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:16.272524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:16.272524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:16.272524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:16.272524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:16.272524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:16.272524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:16.272524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:16.272524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:16.272524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:16.272524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:16.272524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:16.272524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:16.272524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:16.273555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:16.274501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:16.274501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:16.274501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:16.274501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:16.274501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 262/509] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:16.274877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a114cea-d7da-44ae-b9c2-80043d48f439 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:16.275165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a114cea-d7da-44ae-b9c2-80043d48f439 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:16.275443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a114cea-d7da-44ae-b9c2-80043d48f439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:16.293889 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-2a114cea-d7da-44ae-b9c2-80043d48f439 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:16.295091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2a114cea-d7da-44ae-b9c2-80043d48f439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:16.330051 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:16.361039 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 263/510] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:16.362013 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a114cea-d7da-44ae-b9c2-80043d48f439 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:16.372086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d73379-8162-45be-ae38-363ba7dac3e2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:16.407433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201af96-5c7b-4b15-af28-94115eb87c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:16.408104 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201af96-5c7b-4b15-af28-94115eb87c79 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:16.408390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201af96-5c7b-4b15-af28-94115eb87c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:16.408667 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201af96-5c7b-4b15-af28-94115eb87c79 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:16.409453 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201af96-5c7b-4b15-af28-94115eb87c79 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:16.409763 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201af96-5c7b-4b15-af28-94115eb87c79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:16.411393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201af96-5c7b-4b15-af28-94115eb87c79 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:16.419328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 248/511] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:16.420118 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201af96-5c7b-4b15-af28-94115eb87c79 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:16.473978 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a114cea-d7da-44ae-b9c2-80043d48f439 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:16.493392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:16.508367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f6b08-7812-45ad-8674-056578ca077c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:16.509022 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f6b08-7812-45ad-8674-056578ca077c None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:16.509329 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f6b08-7812-45ad-8674-056578ca077c None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:16.509605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f6b08-7812-45ad-8674-056578ca077c None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:16.510445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f6b08-7812-45ad-8674-056578ca077c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:16.510755 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f6b08-7812-45ad-8674-056578ca077c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:16.512191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f6b08-7812-45ad-8674-056578ca077c None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:16.517614 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201af96-5c7b-4b15-af28-94115eb87c79 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:16.529896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 264/512] 10.222.0.42 () {56 vars in 1137 bytes} [Tue Jul 19 15:17:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:16.530694 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f6b08-7812-45ad-8674-056578ca077c None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:16.539668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:16.554258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042b94f-1606-4003-9aad-3ed2d1ce4ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:16.554915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042b94f-1606-4003-9aad-3ed2d1ce4ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:16.555204 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042b94f-1606-4003-9aad-3ed2d1ce4ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:16.555487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042b94f-1606-4003-9aad-3ed2d1ce4ee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:16.623257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042b94f-1606-4003-9aad-3ed2d1ce4ee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:16.624708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042b94f-1606-4003-9aad-3ed2d1ce4ee4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:16.628160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f6b08-7812-45ad-8674-056578ca077c None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:16.662548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9215ee5-c488-4561-9b7d-d41ad57fd94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:16.663200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9215ee5-c488-4561-9b7d-d41ad57fd94d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:16.663480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9215ee5-c488-4561-9b7d-d41ad57fd94d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:16.663755 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9215ee5-c488-4561-9b7d-d41ad57fd94d None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:16.664473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9215ee5-c488-4561-9b7d-d41ad57fd94d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:16.664775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9215ee5-c488-4561-9b7d-d41ad57fd94d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:16.666171 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9215ee5-c488-4561-9b7d-d41ad57fd94d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:16.677720 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 265/513] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:16.678521 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9215ee5-c488-4561-9b7d-d41ad57fd94d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:16.717742 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:16.727126 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 249/514] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:17:16.727898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042b94f-1606-4003-9aad-3ed2d1ce4ee4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:16.748196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:16.781771 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9215ee5-c488-4561-9b7d-d41ad57fd94d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:16.816465 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f718c-c8ef-4540-b550-10b4306756cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:16.817285 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f718c-c8ef-4540-b550-10b4306756cb None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:16.817569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f718c-c8ef-4540-b550-10b4306756cb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:16.817866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f718c-c8ef-4540-b550-10b4306756cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:16.825408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042b94f-1606-4003-9aad-3ed2d1ce4ee4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:16.827964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f718c-c8ef-4540-b550-10b4306756cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:16.829458 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f718c-c8ef-4540-b550-10b4306756cb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:16.847243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-616f718c-c8ef-4540-b550-10b4306756cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:16.847243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:16.847243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:16.847243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:16.847243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:16.847243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:16.847243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:16.847243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:16.847243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:16.847243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:16.847243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:16.847243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:16.847243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:16.847243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:16.847243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:16.848903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:16.849819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:16.849819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:16.849819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:16.849819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:16.849819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 266/515] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:16.850089 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f718c-c8ef-4540-b550-10b4306756cb None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:16.863364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1279255-a1b8-461b-a3d5-85fc48ffffae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:16.864162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1279255-a1b8-461b-a3d5-85fc48ffffae None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:16.864444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1279255-a1b8-461b-a3d5-85fc48ffffae None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:16.864718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1279255-a1b8-461b-a3d5-85fc48ffffae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:16.874860 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1279255-a1b8-461b-a3d5-85fc48ffffae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:16.876325 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1279255-a1b8-461b-a3d5-85fc48ffffae None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:16.893482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c1279255-a1b8-461b-a3d5-85fc48ffffae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:16.893482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:16.893482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:16.893482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:16.893482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:16.893482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:16.893482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:16.893482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:16.893482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:16.893482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:16.893482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:16.893482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:16.893482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:16.893482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:16.893482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:16.894345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:16.895746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:16.895746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:16.895746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:16.895746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:16.895746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 250/516] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:16.896032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1279255-a1b8-461b-a3d5-85fc48ffffae None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:16.944952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:16.955162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f718c-c8ef-4540-b550-10b4306756cb None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:16.966555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:16.992275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29880074-a97b-499c-a00b-c5ed6de5d2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:16.992854 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29880074-a97b-499c-a00b-c5ed6de5d2ca None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:16.993157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29880074-a97b-499c-a00b-c5ed6de5d2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:16.993453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29880074-a97b-499c-a00b-c5ed6de5d2ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:16.996088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1279255-a1b8-461b-a3d5-85fc48ffffae None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:17.029579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34c421-0cc1-4737-968d-0685a2a69e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:17.030401 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34c421-0cc1-4737-968d-0685a2a69e38 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:17.030687 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34c421-0cc1-4737-968d-0685a2a69e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:17.030964 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34c421-0cc1-4737-968d-0685a2a69e38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:17.038259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34c421-0cc1-4737-968d-0685a2a69e38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:17.039714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34c421-0cc1-4737-968d-0685a2a69e38 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:17.053931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-7b34c421-0cc1-4737-968d-0685a2a69e38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:17.053931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:17.053931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:17.053931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:17.053931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:17.053931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:17.053931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:17.053931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:17.053931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:17.053931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:17.053931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:17.053931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:17.053931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:17.053931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:17.053931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:17.054934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:17.060760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:17.060760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:17.060760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:17.060760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:17.060760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:17.060760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:17.060760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:17.060760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:17.060760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 251/517] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:17.060760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34c421-0cc1-4737-968d-0685a2a69e38 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:17.061326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29880074-a97b-499c-a00b-c5ed6de5d2ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:17.062441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29880074-a97b-499c-a00b-c5ed6de5d2ca None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:17.159728 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34c421-0cc1-4737-968d-0685a2a69e38 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:17.160435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:17.170859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 267/518] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:17:17.171667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29880074-a97b-499c-a00b-c5ed6de5d2ca None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:17.193914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:17.199874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3098c-7864-4636-94a5-713bbc99b894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:17.200661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3098c-7864-4636-94a5-713bbc99b894 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:17.200943 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3098c-7864-4636-94a5-713bbc99b894 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:17.201245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3098c-7864-4636-94a5-713bbc99b894 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:17.212481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3098c-7864-4636-94a5-713bbc99b894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:17.213975 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3098c-7864-4636-94a5-713bbc99b894 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:17.230997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9ca3098c-7864-4636-94a5-713bbc99b894 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:17.230997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:17.230997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:17.230997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:17.230997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:17.230997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:17.230997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:17.230997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:17.230997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:17.230997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:17.230997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:17.230997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:17.230997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:17.230997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:17.230997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:17.231880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:17.232715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:17.232715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:17.232715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:17.232715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:17.232715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 252/519] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:17.233331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3098c-7864-4636-94a5-713bbc99b894 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:17.278683 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29880074-a97b-499c-a00b-c5ed6de5d2ca None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:17.313217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541c325-5560-4592-bf85-239bbd63aab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:17.313992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541c325-5560-4592-bf85-239bbd63aab0 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:17.314279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541c325-5560-4592-bf85-239bbd63aab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:17.314559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541c325-5560-4592-bf85-239bbd63aab0 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:17.315288 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541c325-5560-4592-bf85-239bbd63aab0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:17.315597 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541c325-5560-4592-bf85-239bbd63aab0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:17.317102 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541c325-5560-4592-bf85-239bbd63aab0 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:17.328203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 268/520] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:17.329074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541c325-5560-4592-bf85-239bbd63aab0 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:17.335965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3098c-7864-4636-94a5-713bbc99b894 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:17.370006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662e837-7b79-49b3-908d-14b571ad7385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:17.370668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662e837-7b79-49b3-908d-14b571ad7385 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:17.370956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662e837-7b79-49b3-908d-14b571ad7385 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:17.371238 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662e837-7b79-49b3-908d-14b571ad7385 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:17.371968 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662e837-7b79-49b3-908d-14b571ad7385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:17.372280 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662e837-7b79-49b3-908d-14b571ad7385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:17.373684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662e837-7b79-49b3-908d-14b571ad7385 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:17.381747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:17.387080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 253/521] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:17.388053 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662e837-7b79-49b3-908d-14b571ad7385 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:17.436779 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541c325-5560-4592-bf85-239bbd63aab0 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:17.439124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:17.470482 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe9cc9-2ee0-406c-bdb5-1e48fe0bace7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:17.471145 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe9cc9-2ee0-406c-bdb5-1e48fe0bace7 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:17.471432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe9cc9-2ee0-406c-bdb5-1e48fe0bace7 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:17.471704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe9cc9-2ee0-406c-bdb5-1e48fe0bace7 None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:17.472573 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe9cc9-2ee0-406c-bdb5-1e48fe0bace7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:17.472894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe9cc9-2ee0-406c-bdb5-1e48fe0bace7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:17.474526 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe9cc9-2ee0-406c-bdb5-1e48fe0bace7 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:17.492697 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 269/522] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:17.493492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662e837-7b79-49b3-908d-14b571ad7385 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:17.493614 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe9cc9-2ee0-406c-bdb5-1e48fe0bace7 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:17.531541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef72f7b-9ee9-4e49-a030-dee64b877a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:17.532259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef72f7b-9ee9-4e49-a030-dee64b877a38 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:17.532543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef72f7b-9ee9-4e49-a030-dee64b877a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:17.532819 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef72f7b-9ee9-4e49-a030-dee64b877a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:17.542979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef72f7b-9ee9-4e49-a030-dee64b877a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:17.544413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef72f7b-9ee9-4e49-a030-dee64b877a38 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:17.562522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-8ef72f7b-9ee9-4e49-a030-dee64b877a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:17.562522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:17.562522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:17.562522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:17.562522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:17.562522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:17.562522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:17.562522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:17.562522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:17.562522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:17.562522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:17.562522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:17.562522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:17.562522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:17.562522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:17.563381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:17.564239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:17.564239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:17.564239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:17.564239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:17.564239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 254/523] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:17.564645 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef72f7b-9ee9-4e49-a030-dee64b877a38 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:17.595517 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe9cc9-2ee0-406c-bdb5-1e48fe0bace7 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:17.606203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:17.630610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b63ab9-bf0e-43ac-876b-f856ec0344df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:17.631273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b63ab9-bf0e-43ac-876b-f856ec0344df None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:17.631556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b63ab9-bf0e-43ac-876b-f856ec0344df None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:17.631834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b63ab9-bf0e-43ac-876b-f856ec0344df None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:17.632560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b63ab9-bf0e-43ac-876b-f856ec0344df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:17.632866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b63ab9-bf0e-43ac-876b-f856ec0344df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:17.634295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b63ab9-bf0e-43ac-876b-f856ec0344df None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:17.646909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 270/524] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:17.647703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b63ab9-bf0e-43ac-876b-f856ec0344df None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:17.661622 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:17.667862 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef72f7b-9ee9-4e49-a030-dee64b877a38 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:17.701842 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d07ca4-d489-4ae5-9697-9cb75a646d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:17.702505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d07ca4-d489-4ae5-9697-9cb75a646d1f None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:17.702788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d07ca4-d489-4ae5-9697-9cb75a646d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:17.703061 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d07ca4-d489-4ae5-9697-9cb75a646d1f None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:17.703777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d07ca4-d489-4ae5-9697-9cb75a646d1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:17.705211 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d07ca4-d489-4ae5-9697-9cb75a646d1f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:17.725600 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 255/525] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:17:17 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:17:17.733347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d07ca4-d489-4ae5-9697-9cb75a646d1f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:17.758715 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b63ab9-bf0e-43ac-876b-f856ec0344df None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:17.798215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0685383-a7dd-415a-a7e0-0b2cfe51f6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:17.798937 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0685383-a7dd-415a-a7e0-0b2cfe51f6aa None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:17.799214 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0685383-a7dd-415a-a7e0-0b2cfe51f6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:17.799495 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0685383-a7dd-415a-a7e0-0b2cfe51f6aa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:17.808399 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0685383-a7dd-415a-a7e0-0b2cfe51f6aa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:17.809953 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0685383-a7dd-415a-a7e0-0b2cfe51f6aa None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-a0685383-a7dd-415a-a7e0-0b2cfe51f6aa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:17.827166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:17.828165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:17.829641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:17.829641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:17.829641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:17.829641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:17.829641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:17.829641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:17.829641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:17.829641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:17.829641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 15:17:17.829641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:17.829641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 271/526] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:17.829641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0685383-a7dd-415a-a7e0-0b2cfe51f6aa None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:17.845478 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d07ca4-d489-4ae5-9697-9cb75a646d1f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:17.866319 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:17.886175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f2c05-d716-4268-aa71-512b49b31d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:17.886846 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f2c05-d716-4268-aa71-512b49b31d32 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:17.887135 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f2c05-d716-4268-aa71-512b49b31d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:17.887412 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f2c05-d716-4268-aa71-512b49b31d32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:17.888386 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f2c05-d716-4268-aa71-512b49b31d32 None admin] RBAC: Adding query filter params (role.id=c8578ede4e7041ecaeba79c255cc650b, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=bff6a2d4bc734904b2615608a86b097d) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:17.888583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f2c05-d716-4268-aa71-512b49b31d32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:17.890062 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f2c05-d716-4268-aa71-512b49b31d32 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:17.896339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 256/527] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:17 2022] GET /identity/v3/role_assignments?user.id=bff6a2d4bc734904b2615608a86b097d&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=c8578ede4e7041ecaeba79c255cc650b => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:17.897245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f2c05-d716-4268-aa71-512b49b31d32 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:17.926145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:17.961344 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0685383-a7dd-415a-a7e0-0b2cfe51f6aa None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:18.012876 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc9c28-49ea-4c3f-a50f-77c3f91dc67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:18.014385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc9c28-49ea-4c3f-a50f-77c3f91dc67d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:18.015092 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc9c28-49ea-4c3f-a50f-77c3f91dc67d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:18.015540 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc9c28-49ea-4c3f-a50f-77c3f91dc67d None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:18.016688 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc9c28-49ea-4c3f-a50f-77c3f91dc67d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:18.017589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc9c28-49ea-4c3f-a50f-77c3f91dc67d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:18.019156 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc9c28-49ea-4c3f-a50f-77c3f91dc67d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:18.037037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f2c05-d716-4268-aa71-512b49b31d32 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:18.041302 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 272/528] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:18.042398 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc9c28-49ea-4c3f-a50f-77c3f91dc67d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:18.086897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438bf6d-4235-4de9-a3c4-5a3506f9a6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:18.087629 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438bf6d-4235-4de9-a3c4-5a3506f9a6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:18.087914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438bf6d-4235-4de9-a3c4-5a3506f9a6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:18.088194 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438bf6d-4235-4de9-a3c4-5a3506f9a6e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:18.096736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438bf6d-4235-4de9-a3c4-5a3506f9a6e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:18.099267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438bf6d-4235-4de9-a3c4-5a3506f9a6e7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:18.117671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e438bf6d-4235-4de9-a3c4-5a3506f9a6e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:18.117671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:18.117671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:18.117671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:18.117671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:18.117671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:18.117671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:18.117671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:18.117671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:18.117671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:18.117671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:18.117671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:18.117671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:18.117671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:18.117671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:18.118732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:18.119764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:18.119764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:18.119764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:18.119764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:18.119764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:18.119764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:18.119764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:18.119764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:18.119764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 257/529] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:18.120234 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438bf6d-4235-4de9-a3c4-5a3506f9a6e7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:18.164841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:18.176126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc9c28-49ea-4c3f-a50f-77c3f91dc67d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:18.208818 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:18.216426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792ec49-bb49-4cfb-aade-c0cedc9308f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:18.217396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792ec49-bb49-4cfb-aade-c0cedc9308f2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:18.217867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792ec49-bb49-4cfb-aade-c0cedc9308f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:18.218426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792ec49-bb49-4cfb-aade-c0cedc9308f2 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:18.219541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792ec49-bb49-4cfb-aade-c0cedc9308f2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:18.220047 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792ec49-bb49-4cfb-aade-c0cedc9308f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:18.221682 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792ec49-bb49-4cfb-aade-c0cedc9308f2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:18.235304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438bf6d-4235-4de9-a3c4-5a3506f9a6e7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:18.245595 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 273/530] 10.222.0.42 () {56 vars in 1139 bytes} [Tue Jul 19 15:17:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:18.246673 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792ec49-bb49-4cfb-aade-c0cedc9308f2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:18.277874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb55742-bf7c-4037-b4fc-0f40a696fc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:18.278768 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb55742-bf7c-4037-b4fc-0f40a696fc1c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:18.279123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb55742-bf7c-4037-b4fc-0f40a696fc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:18.279511 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb55742-bf7c-4037-b4fc-0f40a696fc1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:18.288934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb55742-bf7c-4037-b4fc-0f40a696fc1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:18.290609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb55742-bf7c-4037-b4fc-0f40a696fc1c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:18.305608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-4eb55742-bf7c-4037-b4fc-0f40a696fc1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:18.305608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:18.305608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:18.305608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:18.305608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:18.305608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:18.305608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:18.305608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:18.305608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:18.305608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:18.305608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:18.305608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:18.305608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:18.305608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:18.305608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:18.306661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:18.307715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:18.307715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:18.307715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:18.307715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:18.307715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:18.307715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:18.307715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:18.307715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:18.309188 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 258/531] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:18.310139 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb55742-bf7c-4037-b4fc-0f40a696fc1c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:18.369259 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792ec49-bb49-4cfb-aade-c0cedc9308f2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:18.406248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3dc65-7d5e-429e-8237-0587029f24da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:18.407178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3dc65-7d5e-429e-8237-0587029f24da None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:18.407607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3dc65-7d5e-429e-8237-0587029f24da None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:18.408026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3dc65-7d5e-429e-8237-0587029f24da None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:18.408964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3dc65-7d5e-429e-8237-0587029f24da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:18.409441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3dc65-7d5e-429e-8237-0587029f24da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:18.411309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3dc65-7d5e-429e-8237-0587029f24da None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:18.428529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb55742-bf7c-4037-b4fc-0f40a696fc1c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:18.429211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 274/532] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:18.430241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3dc65-7d5e-429e-8237-0587029f24da None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:18.442461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:18.468554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca3dc6-f6fd-4ce0-8d6d-1d3b02087daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:18.469286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca3dc6-f6fd-4ce0-8d6d-1d3b02087daf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:18.470270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca3dc6-f6fd-4ce0-8d6d-1d3b02087daf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:18.470270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca3dc6-f6fd-4ce0-8d6d-1d3b02087daf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:18.473853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:18.540177 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3dc65-7d5e-429e-8237-0587029f24da None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:18.548555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca3dc6-f6fd-4ce0-8d6d-1d3b02087daf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:18.550274 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca3dc6-f6fd-4ce0-8d6d-1d3b02087daf None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:18.576130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252cf44-b866-4947-bdb2-68f2a7299f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:18.577037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252cf44-b866-4947-bdb2-68f2a7299f07 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:18.577491 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252cf44-b866-4947-bdb2-68f2a7299f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:18.577939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252cf44-b866-4947-bdb2-68f2a7299f07 None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:18.578880 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252cf44-b866-4947-bdb2-68f2a7299f07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:18.579333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252cf44-b866-4947-bdb2-68f2a7299f07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:18.580913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252cf44-b866-4947-bdb2-68f2a7299f07 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:18.594109 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 275/533] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:18.595139 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252cf44-b866-4947-bdb2-68f2a7299f07 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:18.665664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:18.668669 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 259/534] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:17:18.670181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca3dc6-f6fd-4ce0-8d6d-1d3b02087daf None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:18.701939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252cf44-b866-4947-bdb2-68f2a7299f07 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:18.708772 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:18.739625 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751221f3-3e1a-4941-a454-1e7cef8bab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:18.740283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751221f3-3e1a-4941-a454-1e7cef8bab49 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:18.740565 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751221f3-3e1a-4941-a454-1e7cef8bab49 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:18.741042 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751221f3-3e1a-4941-a454-1e7cef8bab49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:18.741735 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751221f3-3e1a-4941-a454-1e7cef8bab49 None admin] RBAC: Adding query filter params (role.id=c8578ede4e7041ecaeba79c255cc650b, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=215c6f6f1aad486f85ce20590d889cbc) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:18.742028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751221f3-3e1a-4941-a454-1e7cef8bab49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:18.743463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751221f3-3e1a-4941-a454-1e7cef8bab49 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:18.757501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 276/535] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:18 2022] GET /identity/v3/role_assignments?user.id=215c6f6f1aad486f85ce20590d889cbc&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=c8578ede4e7041ecaeba79c255cc650b => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:18.758793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751221f3-3e1a-4941-a454-1e7cef8bab49 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:18.785698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca3dc6-f6fd-4ce0-8d6d-1d3b02087daf None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:18.826131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219d0f4-c77c-4e1b-8ff7-a996f509c64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:18.827112 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219d0f4-c77c-4e1b-8ff7-a996f509c64c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:18.827563 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219d0f4-c77c-4e1b-8ff7-a996f509c64c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:18.827989 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219d0f4-c77c-4e1b-8ff7-a996f509c64c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:18.829018 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219d0f4-c77c-4e1b-8ff7-a996f509c64c None admin] RBAC: Adding query filter params (role.id=c8578ede4e7041ecaeba79c255cc650b, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=7245ef27d2454927a01a7cd56a4c27aa) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:18.829515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219d0f4-c77c-4e1b-8ff7-a996f509c64c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:18.831157 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219d0f4-c77c-4e1b-8ff7-a996f509c64c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:18.838481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 260/536] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:18 2022] GET /identity/v3/role_assignments?user.id=7245ef27d2454927a01a7cd56a4c27aa&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=c8578ede4e7041ecaeba79c255cc650b => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:18.960936 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751221f3-3e1a-4941-a454-1e7cef8bab49 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:18.998754 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:19.004205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47daee48-1d1e-41cf-a1dc-1eda0ae0f2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:19.005405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daee48-1d1e-41cf-a1dc-1eda0ae0f2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:19.005913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daee48-1d1e-41cf-a1dc-1eda0ae0f2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:19.006470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daee48-1d1e-41cf-a1dc-1eda0ae0f2b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:19.017928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47daee48-1d1e-41cf-a1dc-1eda0ae0f2b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:19.020255 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47daee48-1d1e-41cf-a1dc-1eda0ae0f2b9 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:19.038282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-47daee48-1d1e-41cf-a1dc-1eda0ae0f2b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:19.038282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:19.038282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:19.038282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:19.038282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:19.038282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:19.038282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:19.038282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:19.038282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:19.038282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:19.038282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:19.038282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:19.038282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:19.038282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:19.038282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:19.039181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:19.040210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:19.040210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:19.040210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:19.040210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:19.040210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 277/537] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:19.044979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219d0f4-c77c-4e1b-8ff7-a996f509c64c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:19.197773 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:19.200742 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219d0f4-c77c-4e1b-8ff7-a996f509c64c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:19.248720 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812c7bb-5575-4d5b-be8b-fa50e08ba13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:19.249417 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812c7bb-5575-4d5b-be8b-fa50e08ba13f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:19.249704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812c7bb-5575-4d5b-be8b-fa50e08ba13f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:19.249978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812c7bb-5575-4d5b-be8b-fa50e08ba13f None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:19.250708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812c7bb-5575-4d5b-be8b-fa50e08ba13f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:19.251017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812c7bb-5575-4d5b-be8b-fa50e08ba13f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:19.252374 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812c7bb-5575-4d5b-be8b-fa50e08ba13f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:19.272988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 261/538] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:19.278269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47daee48-1d1e-41cf-a1dc-1eda0ae0f2b9 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:19.406177 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47daee48-1d1e-41cf-a1dc-1eda0ae0f2b9 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:19.460326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed637666-2d2e-4fca-9dc8-db7d595b04d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:19.461342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed637666-2d2e-4fca-9dc8-db7d595b04d4 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:19.461839 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed637666-2d2e-4fca-9dc8-db7d595b04d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:19.462271 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed637666-2d2e-4fca-9dc8-db7d595b04d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:19.474606 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed637666-2d2e-4fca-9dc8-db7d595b04d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:19.476156 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed637666-2d2e-4fca-9dc8-db7d595b04d4 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:19.497712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-ed637666-2d2e-4fca-9dc8-db7d595b04d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:19.497712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:19.497712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:19.497712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:19.497712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:19.497712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:19.497712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:19.497712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:19.497712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:19.497712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:19.497712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:19.497712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:19.497712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:19.497712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:19.497712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:19.498965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:19.500693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:19.500693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:19.500693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:19.500693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:19.500693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:19.500693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:19.500693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:19.500693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:19.500693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 278/539] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:19.505511 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812c7bb-5575-4d5b-be8b-fa50e08ba13f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:19.636055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812c7bb-5575-4d5b-be8b-fa50e08ba13f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:19.677700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d162d04-f8b8-4c32-b542-5b2721a66375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:19.678373 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d162d04-f8b8-4c32-b542-5b2721a66375 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:19.678661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d162d04-f8b8-4c32-b542-5b2721a66375 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:19.678936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d162d04-f8b8-4c32-b542-5b2721a66375 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:19.679733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d162d04-f8b8-4c32-b542-5b2721a66375 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:19.680040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d162d04-f8b8-4c32-b542-5b2721a66375 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:19.681489 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d162d04-f8b8-4c32-b542-5b2721a66375 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:19.708965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 262/540] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:19.714945 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed637666-2d2e-4fca-9dc8-db7d595b04d4 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:19.775585 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:19.849943 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed637666-2d2e-4fca-9dc8-db7d595b04d4 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:19.890235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b111c7-d605-41ce-a3de-ecd1f482f24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:19.890888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b111c7-d605-41ce-a3de-ecd1f482f24b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:19.891167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b111c7-d605-41ce-a3de-ecd1f482f24b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:19.891442 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b111c7-d605-41ce-a3de-ecd1f482f24b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:19.892276 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b111c7-d605-41ce-a3de-ecd1f482f24b None admin] RBAC: Adding query filter params (role.id=c8578ede4e7041ecaeba79c255cc650b, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=369bdaded0534142b03ad3cf43a70881) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:19.892585 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b111c7-d605-41ce-a3de-ecd1f482f24b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:19.894125 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b111c7-d605-41ce-a3de-ecd1f482f24b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:19.900512 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 279/541] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:19 2022] GET /identity/v3/role_assignments?user.id=369bdaded0534142b03ad3cf43a70881&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=c8578ede4e7041ecaeba79c255cc650b => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:20.125087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0ba68-4b78-41df-a1e7-66445720b20a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:20.125592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0ba68-4b78-41df-a1e7-66445720b20a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:20.125592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0ba68-4b78-41df-a1e7-66445720b20a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:20.127265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 263/542] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:20.133361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa7178-fc53-4b27-ad55-cd638cc2f8aa None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:20.133790 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa7178-fc53-4b27-ad55-cd638cc2f8aa None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:20.133967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa7178-fc53-4b27-ad55-cd638cc2f8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:20.151549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-cefa7178-fc53-4b27-ad55-cd638cc2f8aa None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:20.152735 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cefa7178-fc53-4b27-ad55-cd638cc2f8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:20.171778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30468e61-ed6b-4dbe-afa8-a023cd3058b2 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:20.172082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30468e61-ed6b-4dbe-afa8-a023cd3058b2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:20.172337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30468e61-ed6b-4dbe-afa8-a023cd3058b2 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:20.174158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 264/543] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:17:20.179284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e307b7d-cd45-4e87-b295-086e39f6dcd6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:20.179571 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e307b7d-cd45-4e87-b295-086e39f6dcd6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:20.179854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e307b7d-cd45-4e87-b295-086e39f6dcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:20.197645 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6e307b7d-cd45-4e87-b295-086e39f6dcd6 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:20.198836 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6e307b7d-cd45-4e87-b295-086e39f6dcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:20.214515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:20.250556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 280/544] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:17:20.275960 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa7178-fc53-4b27-ad55-cd638cc2f8aa None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:20.310704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 265/545] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:17:20.335397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e307b7d-cd45-4e87-b295-086e39f6dcd6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:20.339299 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:20.409378 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa7178-fc53-4b27-ad55-cd638cc2f8aa None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:20.449558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8c152-6f92-4b60-ab55-6a4f25024bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:20.450360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8c152-6f92-4b60-ab55-6a4f25024bef None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:20.450637 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8c152-6f92-4b60-ab55-6a4f25024bef None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:20.450909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8c152-6f92-4b60-ab55-6a4f25024bef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:20.469731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8c152-6f92-4b60-ab55-6a4f25024bef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-4ea8c152-6f92-4b60-ab55-6a4f25024bef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:20.480761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:20.481738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:20.482560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:20.482560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:20.482560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:20.482560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:20.482560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:20.482560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:20.482560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:20.482560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:20.482560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:20.482560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:20.482560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:20.482560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:20.482560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 281/546] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:20.490487 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8c152-6f92-4b60-ab55-6a4f25024bef None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:20.512902 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e307b7d-cd45-4e87-b295-086e39f6dcd6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:20.600599 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8c152-6f92-4b60-ab55-6a4f25024bef None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:20.603665 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:20.617285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd5567-a64b-4d22-926a-ce0993381ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:20.618371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd5567-a64b-4d22-926a-ce0993381ecb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:20.618907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd5567-a64b-4d22-926a-ce0993381ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:20.619388 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd5567-a64b-4d22-926a-ce0993381ecb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:20.641165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095331e9-a671-4ff7-a453-92c59755a095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:20.641165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095331e9-a671-4ff7-a453-92c59755a095 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:20.641165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095331e9-a671-4ff7-a453-92c59755a095 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:20.641165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095331e9-a671-4ff7-a453-92c59755a095 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:20.641568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095331e9-a671-4ff7-a453-92c59755a095 None admin] RBAC: Adding query filter params (name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:20.641568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095331e9-a671-4ff7-a453-92c59755a095 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:20.641568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095331e9-a671-4ff7-a453-92c59755a095 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:20.645400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd5567-a64b-4d22-926a-ce0993381ecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:20.649196 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 282/547] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c0fd5567-a64b-4d22-926a-ce0993381ecb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:20.658621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:20.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:20.660550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:20.660550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:20.660550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:20.660550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:20.660550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:20.660550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:20.660550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:20.660550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:20.660550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:20.660550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:20.660550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:20.660550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:20.663608 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095331e9-a671-4ff7-a453-92c59755a095 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:20.664794 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 266/548] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:20.668585 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd5567-a64b-4d22-926a-ce0993381ecb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:20.688174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:20.842860 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd5567-a64b-4d22-926a-ce0993381ecb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:20.846475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095331e9-a671-4ff7-a453-92c59755a095 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:20.903718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4207c6a-6f92-4a0f-b8b4-6f2f9ee0e16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:20.905537 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4207c6a-6f92-4a0f-b8b4-6f2f9ee0e16b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:20.906502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4207c6a-6f92-4a0f-b8b4-6f2f9ee0e16b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:20.907274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbdb62-537f-4f45-ac43-066a45e835c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:20.907722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4207c6a-6f92-4a0f-b8b4-6f2f9ee0e16b None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:20.908629 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4207c6a-6f92-4a0f-b8b4-6f2f9ee0e16b None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:20.909648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbdb62-537f-4f45-ac43-066a45e835c5 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:20.910746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4207c6a-6f92-4a0f-b8b4-6f2f9ee0e16b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:20.911174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbdb62-537f-4f45-ac43-066a45e835c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:20.911595 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbdb62-537f-4f45-ac43-066a45e835c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:20.913168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4207c6a-6f92-4a0f-b8b4-6f2f9ee0e16b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:20.932122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 267/549] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:20.933408 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbdb62-537f-4f45-ac43-066a45e835c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:20.935032 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbdb62-537f-4f45-ac43-066a45e835c5 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:20.937571 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4207c6a-6f92-4a0f-b8b4-6f2f9ee0e16b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:20.968155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-88cbdb62-537f-4f45-ac43-066a45e835c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:20.968155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:20.968155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:20.968155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:20.968155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:20.968155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:20.968155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:20.968155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:20.968155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:20.968155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:20.968155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:20.968155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:20.968155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:20.968155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:20.968155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:20.969772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:20.971025 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:20.971025 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:20.971025 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:20.971025 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:20.980444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 283/550] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:20.987054 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbdb62-537f-4f45-ac43-066a45e835c5 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:21.013486 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:21.085528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:21.142634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4207c6a-6f92-4a0f-b8b4-6f2f9ee0e16b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:21.176395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbdb62-537f-4f45-ac43-066a45e835c5 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:21.212939 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b19de7-f6e2-4b83-abce-28926b06b6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:21.214052 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b19de7-f6e2-4b83-abce-28926b06b6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:21.215980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b19de7-f6e2-4b83-abce-28926b06b6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:21.220195 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b19de7-f6e2-4b83-abce-28926b06b6b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:21.246359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f82326-48b4-4cf3-9768-3cc5efbc8d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:21.247399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b19de7-f6e2-4b83-abce-28926b06b6b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:21.249050 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b19de7-f6e2-4b83-abce-28926b06b6b4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:21.251928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f82326-48b4-4cf3-9768-3cc5efbc8d32 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:21.253088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f82326-48b4-4cf3-9768-3cc5efbc8d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:21.253719 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f82326-48b4-4cf3-9768-3cc5efbc8d32 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:21.258412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f82326-48b4-4cf3-9768-3cc5efbc8d32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:21.258731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f82326-48b4-4cf3-9768-3cc5efbc8d32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:21.260135 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f82326-48b4-4cf3-9768-3cc5efbc8d32 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:21.279849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-80b19de7-f6e2-4b83-abce-28926b06b6b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:21.279849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:21.279849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:21.279849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:21.279849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:21.279849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:21.279849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:21.279849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:21.279849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:21.279849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:21.279849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:21.279849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:21.279849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:21.279849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:21.279849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:21.280963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:21.282212 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:21.282212 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:21.282212 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:21.282212 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:21.291339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 284/551] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:21.291612 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 268/552] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:21.294404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f82326-48b4-4cf3-9768-3cc5efbc8d32 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:21.301268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b19de7-f6e2-4b83-abce-28926b06b6b4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:21.441645 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:21.464601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f82326-48b4-4cf3-9768-3cc5efbc8d32 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:21.486073 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b19de7-f6e2-4b83-abce-28926b06b6b4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:21.510658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d4b61-d8b2-4d5a-b316-cf1f6f30a296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:21.511725 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:21.512138 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d4b61-d8b2-4d5a-b316-cf1f6f30a296 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:21.513510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d4b61-d8b2-4d5a-b316-cf1f6f30a296 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:21.513962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d4b61-d8b2-4d5a-b316-cf1f6f30a296 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:21.515401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d4b61-d8b2-4d5a-b316-cf1f6f30a296 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:21.515876 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d4b61-d8b2-4d5a-b316-cf1f6f30a296 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:21.517885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d4b61-d8b2-4d5a-b316-cf1f6f30a296 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:21.524534 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884fe6d-61d9-4c8e-a99c-0543c9137044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:21.525383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884fe6d-61d9-4c8e-a99c-0543c9137044 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:21.525670 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884fe6d-61d9-4c8e-a99c-0543c9137044 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:21.525950 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884fe6d-61d9-4c8e-a99c-0543c9137044 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:21.534577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884fe6d-61d9-4c8e-a99c-0543c9137044 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:21.539621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884fe6d-61d9-4c8e-a99c-0543c9137044 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:21.539876 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 285/553] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:21.541635 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdb680-540c-4b3f-b28d-68917d96639b None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:21.542657 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdb680-540c-4b3f-b28d-68917d96639b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:21.543095 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdb680-540c-4b3f-b28d-68917d96639b None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:21.544882 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 286/554] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:21.545779 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdb680-540c-4b3f-b28d-68917d96639b None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d884fe6d-61d9-4c8e-a99c-0543c9137044 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:21.562735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:21.563991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:21.565198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:21.565198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:21.565198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:21.565198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:21.565198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:21.565198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:21.565198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:21.565198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:21.565198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 15:17:21.565198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:21.571253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 269/555] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:17:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:21.571253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f78de-bdb4-43db-b328-e0e60e40b4d1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:21.571253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f78de-bdb4-43db-b328-e0e60e40b4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:21.571253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f78de-bdb4-43db-b328-e0e60e40b4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:21.589797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-564f78de-bdb4-43db-b328-e0e60e40b4d1 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:21.590841 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-564f78de-bdb4-43db-b328-e0e60e40b4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:21.665086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 270/556] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:21.666274 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f78de-bdb4-43db-b328-e0e60e40b4d1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:21.675314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdb680-540c-4b3f-b28d-68917d96639b None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:21.708333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:21.713644 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293186e4-1e2d-47e9-b3d8-c4e905492ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:21.714440 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293186e4-1e2d-47e9-b3d8-c4e905492ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:21.714716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293186e4-1e2d-47e9-b3d8-c4e905492ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:21.715002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293186e4-1e2d-47e9-b3d8-c4e905492ca2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:21.723043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293186e4-1e2d-47e9-b3d8-c4e905492ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:21.724497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293186e4-1e2d-47e9-b3d8-c4e905492ca2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-293186e4-1e2d-47e9-b3d8-c4e905492ca2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:21.741476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:21.742542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:21.743938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:21.743938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:21.743938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:21.743938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:21.743938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:21.743938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:21.743938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:21.743938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:21.743938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:17:21.743938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:21.743938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 287/557] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Jul 19 15:17:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:21.744530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf84e99-1482-4b1c-97e1-dfc9ed6c8bd2 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:21.744530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf84e99-1482-4b1c-97e1-dfc9ed6c8bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:21.744660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf84e99-1482-4b1c-97e1-dfc9ed6c8bd2 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:21.746763 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 288/558] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:21.747544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf84e99-1482-4b1c-97e1-dfc9ed6c8bd2 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:21.794461 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f78de-bdb4-43db-b328-e0e60e40b4d1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:21.808441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:21.830477 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ffee8-bf92-4a5f-ad80-ac9e1a53aa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:21.831155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ffee8-bf92-4a5f-ad80-ac9e1a53aa23 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:21.831460 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ffee8-bf92-4a5f-ad80-ac9e1a53aa23 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:21.831737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ffee8-bf92-4a5f-ad80-ac9e1a53aa23 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:21.832888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ffee8-bf92-4a5f-ad80-ac9e1a53aa23 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:21.833232 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ffee8-bf92-4a5f-ad80-ac9e1a53aa23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:21.834661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ffee8-bf92-4a5f-ad80-ac9e1a53aa23 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:21.852791 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 271/559] 10.222.0.42 () {56 vars in 1135 bytes} [Tue Jul 19 15:17:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:21.853664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ffee8-bf92-4a5f-ad80-ac9e1a53aa23 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:21.873163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf84e99-1482-4b1c-97e1-dfc9ed6c8bd2 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:21.912397 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4c03c-8f50-440e-9e1b-91660c7ed9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:21.913501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4c03c-8f50-440e-9e1b-91660c7ed9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:21.913990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4c03c-8f50-440e-9e1b-91660c7ed9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:21.914454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4c03c-8f50-440e-9e1b-91660c7ed9d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:21.924939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4c03c-8f50-440e-9e1b-91660c7ed9d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-39c4c03c-8f50-440e-9e1b-91660c7ed9d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:21.933380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:21.934423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:21.935291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:21.935291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:21.935291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:21.935291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:21.935291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:21.935291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:21.935291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:21.935291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:21.935291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:21.935291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:21.935291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:21.935291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:21.935291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 289/560] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:21.936896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f2c8d-6087-4e4b-9d2b-10df77010d75 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:21.936896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f2c8d-6087-4e4b-9d2b-10df77010d75 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:21.936896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f2c8d-6087-4e4b-9d2b-10df77010d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:21.954498 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-a74f2c8d-6087-4e4b-9d2b-10df77010d75 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:21.955552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a74f2c8d-6087-4e4b-9d2b-10df77010d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:21.968259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ffee8-bf92-4a5f-ad80-ac9e1a53aa23 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:22.004644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0eb7d2-8813-49e8-ba9e-8c545e0306db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:22.005345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0eb7d2-8813-49e8-ba9e-8c545e0306db None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:22.005623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0eb7d2-8813-49e8-ba9e-8c545e0306db None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:22.005732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:22.005896 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0eb7d2-8813-49e8-ba9e-8c545e0306db None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:22.006788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0eb7d2-8813-49e8-ba9e-8c545e0306db None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:22.007164 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0eb7d2-8813-49e8-ba9e-8c545e0306db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:22.008786 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0eb7d2-8813-49e8-ba9e-8c545e0306db None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:22.028005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 290/561] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:22.028775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f2c8d-6087-4e4b-9d2b-10df77010d75 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:22.040393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 272/562] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Jul 19 15:17:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:22.041955 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0eb7d2-8813-49e8-ba9e-8c545e0306db None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:22.076677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:22.136005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f2c8d-6087-4e4b-9d2b-10df77010d75 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:22.152657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0eb7d2-8813-49e8-ba9e-8c545e0306db None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:22.171445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4497f-0350-4d81-8780-558959936a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:22.172097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4497f-0350-4d81-8780-558959936a9d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:22.172384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4497f-0350-4d81-8780-558959936a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:22.172642 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4497f-0350-4d81-8780-558959936a9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:22.187511 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f370d-a740-488a-b51d-61699cdca83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:22.188181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f370d-a740-488a-b51d-61699cdca83e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:22.188464 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f370d-a740-488a-b51d-61699cdca83e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:22.188741 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f370d-a740-488a-b51d-61699cdca83e None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:22.189514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f370d-a740-488a-b51d-61699cdca83e None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:22.189821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f370d-a740-488a-b51d-61699cdca83e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:22.191242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f370d-a740-488a-b51d-61699cdca83e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:22.199207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 273/563] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:22.199982 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f370d-a740-488a-b51d-61699cdca83e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:22.224130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:22.242758 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4497f-0350-4d81-8780-558959936a9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:22.244253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4497f-0350-4d81-8780-558959936a9d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:22.292975 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:22.302271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f370d-a740-488a-b51d-61699cdca83e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:22.336700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63204b4c-a205-4e19-a56a-812abc69e886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:22.337613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63204b4c-a205-4e19-a56a-812abc69e886 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:22.337897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63204b4c-a205-4e19-a56a-812abc69e886 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:22.338178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63204b4c-a205-4e19-a56a-812abc69e886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:22.344463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 291/564] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:17:22.345386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4497f-0350-4d81-8780-558959936a9d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:22.349427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63204b4c-a205-4e19-a56a-812abc69e886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:22.350880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63204b4c-a205-4e19-a56a-812abc69e886 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:22.368932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-63204b4c-a205-4e19-a56a-812abc69e886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:22.368932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:22.368932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:22.368932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:22.368932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:22.368932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:22.368932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:22.368932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:22.368932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:22.368932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:22.368932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:22.368932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:22.368932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:22.368932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:22.368932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:22.370013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:22.370998 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:22.370998 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:22.370998 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:22.370998 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:22.370998 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 274/565] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:22.371365 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbff761-b892-498f-9cc0-5313d279fcef None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:22.371645 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbff761-b892-498f-9cc0-5313d279fcef None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:22.371917 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbff761-b892-498f-9cc0-5313d279fcef None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:22.373202 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 275/566] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:22.373898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbff761-b892-498f-9cc0-5313d279fcef None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:22.430588 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:22.449577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4497f-0350-4d81-8780-558959936a9d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:22.480275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbff761-b892-498f-9cc0-5313d279fcef None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:22.484626 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ef89b-74ed-48ef-8d05-64be539aee19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:22.485461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ef89b-74ed-48ef-8d05-64be539aee19 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:22.485740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ef89b-74ed-48ef-8d05-64be539aee19 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:22.486010 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ef89b-74ed-48ef-8d05-64be539aee19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:22.495964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ef89b-74ed-48ef-8d05-64be539aee19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-7d0ef89b-74ed-48ef-8d05-64be539aee19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:22.504121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:22.505026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:22.505893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:22.505893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:22.505893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:22.505893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:22.505893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:22.505893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:22.505893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:22.505893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:22.505893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:22.505893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:22.505893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:22.505893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:22.505893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 292/567] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:22.506569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ef89b-74ed-48ef-8d05-64be539aee19 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:22.515691 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5b7e6-7a84-4cb8-88b7-5e2886ae11c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:22.516483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5b7e6-7a84-4cb8-88b7-5e2886ae11c8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:22.516759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5b7e6-7a84-4cb8-88b7-5e2886ae11c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:22.517030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5b7e6-7a84-4cb8-88b7-5e2886ae11c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:22.528406 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5b7e6-7a84-4cb8-88b7-5e2886ae11c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:22.529885 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5b7e6-7a84-4cb8-88b7-5e2886ae11c8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:22.532326 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:22.547408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e6f5b7e6-7a84-4cb8-88b7-5e2886ae11c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:22.547408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:22.547408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:22.547408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:22.547408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:22.547408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:22.547408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:22.547408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:22.547408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:22.547408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:22.547408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:22.547408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:22.547408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:22.547408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:22.547408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:22.548337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:22.549268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:22.549268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:22.549268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:22.549268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:22.549268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 276/568] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:22.549959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5b7e6-7a84-4cb8-88b7-5e2886ae11c8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:22.607057 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ef89b-74ed-48ef-8d05-64be539aee19 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:22.639813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58bf9c-0bbe-4ea2-828d-376939a681e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:22.640594 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58bf9c-0bbe-4ea2-828d-376939a681e4 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:22.640872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58bf9c-0bbe-4ea2-828d-376939a681e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:22.641151 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58bf9c-0bbe-4ea2-828d-376939a681e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:22.649675 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5b7e6-7a84-4cb8-88b7-5e2886ae11c8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:22.651729 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58bf9c-0bbe-4ea2-828d-376939a681e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:22.653197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58bf9c-0bbe-4ea2-828d-376939a681e4 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:22.654879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:22.671576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-8a58bf9c-0bbe-4ea2-828d-376939a681e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:22.671576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:22.671576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:22.671576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:22.671576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:22.671576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:22.671576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:22.671576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:22.671576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:22.671576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:22.671576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:22.671576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:22.671576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:22.671576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:22.671576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:22.672579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:22.673530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:22.673530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:22.673530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:22.673530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:22.673530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 293/569] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:22.674029 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3cfe9-cbad-45dd-b65c-2e2b12aa6857 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:22.674313 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3cfe9-cbad-45dd-b65c-2e2b12aa6857 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:22.674590 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3cfe9-cbad-45dd-b65c-2e2b12aa6857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:22.687993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42f4a6-81d8-4ec0-8402-8984846a07fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:22.688658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42f4a6-81d8-4ec0-8402-8984846a07fc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:22.688943 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42f4a6-81d8-4ec0-8402-8984846a07fc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:22.689253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42f4a6-81d8-4ec0-8402-8984846a07fc None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:22.689980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42f4a6-81d8-4ec0-8402-8984846a07fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:22.690289 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42f4a6-81d8-4ec0-8402-8984846a07fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:22.691727 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-b9e3cfe9-cbad-45dd-b65c-2e2b12aa6857 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:22.692109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42f4a6-81d8-4ec0-8402-8984846a07fc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:22.692921 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b9e3cfe9-cbad-45dd-b65c-2e2b12aa6857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:22.703718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 277/570] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:22.704499 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42f4a6-81d8-4ec0-8402-8984846a07fc None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:22.759915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:22.760074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 294/571] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:22.760656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3cfe9-cbad-45dd-b65c-2e2b12aa6857 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:22.823135 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42f4a6-81d8-4ec0-8402-8984846a07fc None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:22.858130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc2630-d109-4e02-b9bb-07af1835a5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:22.858803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc2630-d109-4e02-b9bb-07af1835a5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:22.859090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc2630-d109-4e02-b9bb-07af1835a5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:22.859373 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc2630-d109-4e02-b9bb-07af1835a5b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:22.859612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3cfe9-cbad-45dd-b65c-2e2b12aa6857 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:22.860136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc2630-d109-4e02-b9bb-07af1835a5b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:22.860449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc2630-d109-4e02-b9bb-07af1835a5b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:22.861885 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc2630-d109-4e02-b9bb-07af1835a5b3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:22.871260 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 278/572] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:22.872075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc2630-d109-4e02-b9bb-07af1835a5b3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:22.888226 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:22.899999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515262df-85f9-4156-b4ed-d5d86282e299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:22.900664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515262df-85f9-4156-b4ed-d5d86282e299 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:22.900953 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515262df-85f9-4156-b4ed-d5d86282e299 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:22.901509 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515262df-85f9-4156-b4ed-d5d86282e299 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:22.902239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515262df-85f9-4156-b4ed-d5d86282e299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:22.902546 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515262df-85f9-4156-b4ed-d5d86282e299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:22.903902 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515262df-85f9-4156-b4ed-d5d86282e299 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:22.916242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 295/573] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:22.917037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515262df-85f9-4156-b4ed-d5d86282e299 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:22.975183 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc2630-d109-4e02-b9bb-07af1835a5b3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:22.992121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:23.010599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e58295-b19d-4374-a630-34acbed420da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:23.011225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e58295-b19d-4374-a630-34acbed420da None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:23.011511 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e58295-b19d-4374-a630-34acbed420da None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:23.011791 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e58295-b19d-4374-a630-34acbed420da None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:23.012514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e58295-b19d-4374-a630-34acbed420da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:23.012823 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e58295-b19d-4374-a630-34acbed420da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:23.014234 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e58295-b19d-4374-a630-34acbed420da None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:23.016844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515262df-85f9-4156-b4ed-d5d86282e299 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:23.026784 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 279/574] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:23.027414 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e58295-b19d-4374-a630-34acbed420da None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:23.055178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca11ac6-54b2-4011-8079-1858bd558e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:23.056046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca11ac6-54b2-4011-8079-1858bd558e08 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:23.056330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca11ac6-54b2-4011-8079-1858bd558e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:23.056599 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca11ac6-54b2-4011-8079-1858bd558e08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:23.063117 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca11ac6-54b2-4011-8079-1858bd558e08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:23.064549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca11ac6-54b2-4011-8079-1858bd558e08 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:23.079296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-3ca11ac6-54b2-4011-8079-1858bd558e08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:23.079296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:23.079296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:23.079296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:23.079296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:23.079296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:23.079296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:23.079296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:23.079296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:23.079296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:23.079296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:23.079296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:23.079296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:23.079296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:23.079296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:23.080359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:23.082097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:23.082097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:23.082097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:23.082097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:23.082097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:23.082097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:23.082097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:23.082097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:23.082097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 296/575] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:23.083178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca11ac6-54b2-4011-8079-1858bd558e08 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:23.126308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e58295-b19d-4374-a630-34acbed420da None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:23.127968 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:23.161843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5a303-f602-40fd-922a-c79ca1a5a70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:23.162689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5a303-f602-40fd-922a-c79ca1a5a70e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:23.162978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5a303-f602-40fd-922a-c79ca1a5a70e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:23.163255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5a303-f602-40fd-922a-c79ca1a5a70e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:23.173325 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5a303-f602-40fd-922a-c79ca1a5a70e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-21e5a303-f602-40fd-922a-c79ca1a5a70e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:23.181010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:23.182131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:23.183009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:23.183009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:23.183009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:23.183009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:23.183009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:23.183009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:23.183009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:23.183009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:23.183009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:23.183009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:23.183009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:23.183009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:23.183009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 280/576] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:23.183688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5a303-f602-40fd-922a-c79ca1a5a70e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:23.186618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca11ac6-54b2-4011-8079-1858bd558e08 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:23.221814 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e213b-ec25-4028-83c5-d4c2f9c4965d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:23.222685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e213b-ec25-4028-83c5-d4c2f9c4965d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:23.222966 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e213b-ec25-4028-83c5-d4c2f9c4965d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:23.223240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e213b-ec25-4028-83c5-d4c2f9c4965d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:23.224757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:23.233469 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e213b-ec25-4028-83c5-d4c2f9c4965d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:23.234943 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e213b-ec25-4028-83c5-d4c2f9c4965d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:23.253167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-bb4e213b-ec25-4028-83c5-d4c2f9c4965d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:23.253167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:23.253167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:23.253167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:23.253167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:23.253167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:23.253167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:23.253167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:23.253167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:23.253167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:23.253167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:23.253167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:23.253167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:23.253167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:23.253167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:23.254679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:23.255588 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:23.255588 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:23.255588 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:23.255588 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:23.255588 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 297/577] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:23.255858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e213b-ec25-4028-83c5-d4c2f9c4965d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:23.289925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5a303-f602-40fd-922a-c79ca1a5a70e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:23.323154 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5035ec-54f4-488b-83f2-571bcab2e881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:23.323972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5035ec-54f4-488b-83f2-571bcab2e881 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:23.324255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5035ec-54f4-488b-83f2-571bcab2e881 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:23.324532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5035ec-54f4-488b-83f2-571bcab2e881 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:23.333466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5035ec-54f4-488b-83f2-571bcab2e881 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:23.334903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5035ec-54f4-488b-83f2-571bcab2e881 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-7e5035ec-54f4-488b-83f2-571bcab2e881 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:23.348783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:23.349892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:23.356028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:23.356028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:23.356028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:23.356028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:23.356028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:23.356028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:23.356028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:23.356028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:23.356028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 19 15:17:23.356028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:23.356028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 281/578] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:23.356611 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:23.356692 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5035ec-54f4-488b-83f2-571bcab2e881 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:23.356988 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e213b-ec25-4028-83c5-d4c2f9c4965d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:23.390319 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989faf6-ec45-4943-87e7-8ead7132cfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:23.391106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989faf6-ec45-4943-87e7-8ead7132cfd2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:23.391391 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989faf6-ec45-4943-87e7-8ead7132cfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:23.391668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989faf6-ec45-4943-87e7-8ead7132cfd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:23.398732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989faf6-ec45-4943-87e7-8ead7132cfd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:23.400192 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989faf6-ec45-4943-87e7-8ead7132cfd2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:23.416544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-b989faf6-ec45-4943-87e7-8ead7132cfd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:23.416544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:23.416544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:23.416544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:23.416544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:23.416544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:23.416544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:23.416544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:23.416544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:23.416544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:23.416544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:23.416544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:23.416544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:23.416544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:23.416544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:23.417663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:23.418650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:23.418650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:23.418650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:23.418650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:23.418650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:23.418650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:23.418650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:23.418650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:23.418650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 298/579] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:23.418650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989faf6-ec45-4943-87e7-8ead7132cfd2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:23.456154 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:23.460923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5035ec-54f4-488b-83f2-571bcab2e881 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:23.498836 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc758e-b44d-4f00-a4ab-2535fb21a767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:23.499502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc758e-b44d-4f00-a4ab-2535fb21a767 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:23.499795 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc758e-b44d-4f00-a4ab-2535fb21a767 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:23.500067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc758e-b44d-4f00-a4ab-2535fb21a767 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:23.500857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc758e-b44d-4f00-a4ab-2535fb21a767 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:23.501362 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc758e-b44d-4f00-a4ab-2535fb21a767 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:23.502621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc758e-b44d-4f00-a4ab-2535fb21a767 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:23.515245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 282/580] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:23.515976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc758e-b44d-4f00-a4ab-2535fb21a767 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:23.529628 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989faf6-ec45-4943-87e7-8ead7132cfd2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:23.563646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed8fb0-0906-4cf8-b759-61e0919cecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:23.564241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed8fb0-0906-4cf8-b759-61e0919cecb2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:23.564524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed8fb0-0906-4cf8-b759-61e0919cecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:23.564800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed8fb0-0906-4cf8-b759-61e0919cecb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:23.565581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed8fb0-0906-4cf8-b759-61e0919cecb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:23.565899 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed8fb0-0906-4cf8-b759-61e0919cecb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:23.567355 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed8fb0-0906-4cf8-b759-61e0919cecb2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:23.575196 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 299/581] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:23.575970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed8fb0-0906-4cf8-b759-61e0919cecb2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:23.594664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:23.617372 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc758e-b44d-4f00-a4ab-2535fb21a767 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:23.654479 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e9dbf-adbc-49fe-9a28-115b761577dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:23.655147 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e9dbf-adbc-49fe-9a28-115b761577dc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:23.655431 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e9dbf-adbc-49fe-9a28-115b761577dc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:23.655707 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e9dbf-adbc-49fe-9a28-115b761577dc None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:23.656432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e9dbf-adbc-49fe-9a28-115b761577dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:23.656741 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e9dbf-adbc-49fe-9a28-115b761577dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:23.658144 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e9dbf-adbc-49fe-9a28-115b761577dc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:23.669802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 283/582] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:23.670592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e9dbf-adbc-49fe-9a28-115b761577dc None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:23.680675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed8fb0-0906-4cf8-b759-61e0919cecb2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:23.690110 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:23.714389 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c973e-ccd6-4fbc-b77c-c45414371363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:23.715050 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c973e-ccd6-4fbc-b77c-c45414371363 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:23.715328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c973e-ccd6-4fbc-b77c-c45414371363 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:23.715601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c973e-ccd6-4fbc-b77c-c45414371363 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:23.716435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c973e-ccd6-4fbc-b77c-c45414371363 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:23.716749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c973e-ccd6-4fbc-b77c-c45414371363 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:23.718206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c973e-ccd6-4fbc-b77c-c45414371363 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:23.735473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 300/583] 10.222.0.42 () {56 vars in 1137 bytes} [Tue Jul 19 15:17:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:23.736244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c973e-ccd6-4fbc-b77c-c45414371363 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:23.775246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e9dbf-adbc-49fe-9a28-115b761577dc None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:23.811271 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:23.811726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd0079-2a7b-4b2d-9e10-6c1cb2e9effe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:23.812803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd0079-2a7b-4b2d-9e10-6c1cb2e9effe None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:23.813093 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd0079-2a7b-4b2d-9e10-6c1cb2e9effe None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:23.813440 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd0079-2a7b-4b2d-9e10-6c1cb2e9effe None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:23.814247 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd0079-2a7b-4b2d-9e10-6c1cb2e9effe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:23.814503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd0079-2a7b-4b2d-9e10-6c1cb2e9effe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:23.816932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd0079-2a7b-4b2d-9e10-6c1cb2e9effe None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:23.829802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 284/584] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:23.830598 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd0079-2a7b-4b2d-9e10-6c1cb2e9effe None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:23.836058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c973e-ccd6-4fbc-b77c-c45414371363 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:23.869499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba9772-ba00-4be5-8ea6-f319a35c7bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:23.870378 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba9772-ba00-4be5-8ea6-f319a35c7bdf None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:23.870659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba9772-ba00-4be5-8ea6-f319a35c7bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:23.870930 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba9772-ba00-4be5-8ea6-f319a35c7bdf None admin] PATH_INFO: `/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/bff6a2d4bc734904b2615608a86b097d/roles/c8578ede4e7041ecaeba79c255cc650b` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:23.885740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba9772-ba00-4be5-8ea6-f319a35c7bdf None admin] RBAC: Authorizing `identity:create_grant(project_id=a5f51e47b8e34bf5ac822d8c93853e30, user_id=bff6a2d4bc734904b2615608a86b097d, role_id=c8578ede4e7041ecaeba79c255cc650b)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:23.887185 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba9772-ba00-4be5-8ea6-f319a35c7bdf None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:23.900223 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 301/585] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:17:23 2022] PUT /identity/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/bff6a2d4bc734904b2615608a86b097d/roles/c8578ede4e7041ecaeba79c255cc650b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:17:23.901280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba9772-ba00-4be5-8ea6-f319a35c7bdf None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:23.922777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:23.953180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd0079-2a7b-4b2d-9e10-6c1cb2e9effe None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:23.991711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abf5e7-102b-4d7e-9375-e84fc3b5f560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:23.992518 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abf5e7-102b-4d7e-9375-e84fc3b5f560 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:23.992799 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abf5e7-102b-4d7e-9375-e84fc3b5f560 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:23.993072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abf5e7-102b-4d7e-9375-e84fc3b5f560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:24.004585 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abf5e7-102b-4d7e-9375-e84fc3b5f560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:24.006192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abf5e7-102b-4d7e-9375-e84fc3b5f560 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:24.017672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba9772-ba00-4be5-8ea6-f319a35c7bdf None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:24.025495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-a8abf5e7-102b-4d7e-9375-e84fc3b5f560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:24.025495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:24.025495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:24.025495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:24.025495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:24.025495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:24.025495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:24.025495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:24.025495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:24.025495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:24.025495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:24.025495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:24.025495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:24.025495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:24.025495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:24.026726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:24.027634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:24.027634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:24.027634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:24.027634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:24.027634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 285/586] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:24.027948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abf5e7-102b-4d7e-9375-e84fc3b5f560 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:24.053919 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:24.068730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e931d-6080-4d2f-aa50-d0cd46db94b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:24.069837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e931d-6080-4d2f-aa50-d0cd46db94b1 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:24.070312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e931d-6080-4d2f-aa50-d0cd46db94b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:24.070590 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e931d-6080-4d2f-aa50-d0cd46db94b1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:24.078044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e931d-6080-4d2f-aa50-d0cd46db94b1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:24.079488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e931d-6080-4d2f-aa50-d0cd46db94b1 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-a01e931d-6080-4d2f-aa50-d0cd46db94b1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:24.094676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:24.095666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:24.096601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:24.096601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:24.096601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:24.096601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:24.096601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:24.096601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:24.096601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:24.096601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:24.096601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 15:17:24.096601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:24.096601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 302/587] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:24.096601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e931d-6080-4d2f-aa50-d0cd46db94b1 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:24.149759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abf5e7-102b-4d7e-9375-e84fc3b5f560 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:24.180073 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:24.187122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e18bec-f9c6-45a9-8ab1-00c83f268e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:24.187923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e18bec-f9c6-45a9-8ab1-00c83f268e7e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:24.188203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e18bec-f9c6-45a9-8ab1-00c83f268e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:24.188475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e18bec-f9c6-45a9-8ab1-00c83f268e7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:24.200081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e18bec-f9c6-45a9-8ab1-00c83f268e7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:24.201554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e18bec-f9c6-45a9-8ab1-00c83f268e7e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:24.218251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e931d-6080-4d2f-aa50-d0cd46db94b1 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:24.219821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-79e18bec-f9c6-45a9-8ab1-00c83f268e7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:24.219821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:24.219821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:24.219821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:24.219821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:24.219821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:24.219821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:24.219821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:24.219821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:24.219821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:24.219821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:24.219821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:24.219821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:24.219821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:24.219821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:24.220693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:24.221652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:24.221652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:24.221652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:24.221652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:24.226396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 286/588] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:24.227167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e18bec-f9c6-45a9-8ab1-00c83f268e7e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:24.261769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40794bfb-7b6b-46bb-bf57-cc35da761d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:24.262432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40794bfb-7b6b-46bb-bf57-cc35da761d53 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:24.262712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40794bfb-7b6b-46bb-bf57-cc35da761d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:24.263014 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40794bfb-7b6b-46bb-bf57-cc35da761d53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:24.263824 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40794bfb-7b6b-46bb-bf57-cc35da761d53 None admin] RBAC: Adding query filter params (role.id=c8578ede4e7041ecaeba79c255cc650b, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=96faaac55bc5408685c68a6177d91971) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:24.264137 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40794bfb-7b6b-46bb-bf57-cc35da761d53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:24.265619 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40794bfb-7b6b-46bb-bf57-cc35da761d53 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:24.273069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 303/589] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:24 2022] GET /identity/v3/role_assignments?user.id=96faaac55bc5408685c68a6177d91971&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=c8578ede4e7041ecaeba79c255cc650b => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:24.274243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40794bfb-7b6b-46bb-bf57-cc35da761d53 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:24.342949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:24.364444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e18bec-f9c6-45a9-8ab1-00c83f268e7e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:24.410713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a0db2-8caa-408c-a6df-3e408d260e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:24.410926 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40794bfb-7b6b-46bb-bf57-cc35da761d53 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:24.411890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a0db2-8caa-408c-a6df-3e408d260e1f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:24.412376 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a0db2-8caa-408c-a6df-3e408d260e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:24.413065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a0db2-8caa-408c-a6df-3e408d260e1f None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:24.414986 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a0db2-8caa-408c-a6df-3e408d260e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:24.415539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a0db2-8caa-408c-a6df-3e408d260e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:24.417192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a0db2-8caa-408c-a6df-3e408d260e1f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:24.436780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 287/590] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:24.436780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a0db2-8caa-408c-a6df-3e408d260e1f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:24.453676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f336d-ba87-4ab0-a009-555fd4d88b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:24.454504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f336d-ba87-4ab0-a009-555fd4d88b0c None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:24.454657 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f336d-ba87-4ab0-a009-555fd4d88b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:24.454896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f336d-ba87-4ab0-a009-555fd4d88b0c None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:24.455749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f336d-ba87-4ab0-a009-555fd4d88b0c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:24.456061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f336d-ba87-4ab0-a009-555fd4d88b0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:24.457567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f336d-ba87-4ab0-a009-555fd4d88b0c None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:24.477565 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 304/591] 10.222.0.42 () {56 vars in 1137 bytes} [Tue Jul 19 15:17:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:24.478478 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f336d-ba87-4ab0-a009-555fd4d88b0c None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:24.493037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:24.576974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a0db2-8caa-408c-a6df-3e408d260e1f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:24.589673 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f336d-ba87-4ab0-a009-555fd4d88b0c None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:24.601367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:24.625790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f39395-477d-4c8c-bf75-058c3bce893b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:24.627108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f39395-477d-4c8c-bf75-058c3bce893b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:24.628025 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f39395-477d-4c8c-bf75-058c3bce893b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:24.628771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f39395-477d-4c8c-bf75-058c3bce893b None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:24.629739 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f39395-477d-4c8c-bf75-058c3bce893b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:24.630253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f39395-477d-4c8c-bf75-058c3bce893b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:24.631829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f39395-477d-4c8c-bf75-058c3bce893b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:24.632702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb218918-5dec-4146-8176-c4fd1d3116fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:24.633674 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb218918-5dec-4146-8176-c4fd1d3116fb None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:24.634033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb218918-5dec-4146-8176-c4fd1d3116fb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:24.634364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb218918-5dec-4146-8176-c4fd1d3116fb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:24.642302 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb218918-5dec-4146-8176-c4fd1d3116fb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:24.644211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb218918-5dec-4146-8176-c4fd1d3116fb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:24.648564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 288/592] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:24.649719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f39395-477d-4c8c-bf75-058c3bce893b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-cb218918-5dec-4146-8176-c4fd1d3116fb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:24.661279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:24.662263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:24.663219 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:24.663219 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:24.663219 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:24.663219 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:24.663219 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:24.663219 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:24.663219 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:24.663219 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:24.663219 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 15:17:24.663219 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:24.663219 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 305/593] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:24.663825 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb218918-5dec-4146-8176-c4fd1d3116fb None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:24.766802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f39395-477d-4c8c-bf75-058c3bce893b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:24.768696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:24.808722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1195d3f-99b2-4bf2-9176-68baba267ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:24.809226 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb218918-5dec-4146-8176-c4fd1d3116fb None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:24.809788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1195d3f-99b2-4bf2-9176-68baba267ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:24.810125 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1195d3f-99b2-4bf2-9176-68baba267ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:24.810462 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1195d3f-99b2-4bf2-9176-68baba267ac9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:24.822338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1195d3f-99b2-4bf2-9176-68baba267ac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:24.824004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1195d3f-99b2-4bf2-9176-68baba267ac9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:24.842473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c1195d3f-99b2-4bf2-9176-68baba267ac9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:24.842473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:24.842473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:24.842473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:24.842473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:24.842473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:24.842473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:24.842473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:24.842473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:24.842473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:24.842473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:24.842473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:24.842473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:24.842473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:24.842473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:24.843416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:24.844318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:24.844318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:24.844318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:24.844318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:24.844318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 289/594] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:24.845220 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1195d3f-99b2-4bf2-9176-68baba267ac9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:24.857876 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584615ec-2a60-4cfa-be57-bdd2cbc1caa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:24.859326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584615ec-2a60-4cfa-be57-bdd2cbc1caa6 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:24.860035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584615ec-2a60-4cfa-be57-bdd2cbc1caa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:24.860511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584615ec-2a60-4cfa-be57-bdd2cbc1caa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:24.871656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584615ec-2a60-4cfa-be57-bdd2cbc1caa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:24.873108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584615ec-2a60-4cfa-be57-bdd2cbc1caa6 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:24.889088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-584615ec-2a60-4cfa-be57-bdd2cbc1caa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:24.889088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:24.889088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:24.889088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:24.889088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:24.889088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:24.889088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:24.889088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:24.889088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:24.889088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:24.889088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:24.889088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:24.889088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:24.889088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:24.889088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:24.889996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:24.890872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:24.890872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:24.890872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:24.890872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:24.890872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:24.890872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:24.890872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:24.890872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:24.890872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 306/595] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:24.891797 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584615ec-2a60-4cfa-be57-bdd2cbc1caa6 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:24.932144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:24.967692 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1195d3f-99b2-4bf2-9176-68baba267ac9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:25.004765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da37ea-5db1-4945-ba50-3a31eabce1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:25.005596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da37ea-5db1-4945-ba50-3a31eabce1da None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:25.006073 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da37ea-5db1-4945-ba50-3a31eabce1da None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:25.006205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da37ea-5db1-4945-ba50-3a31eabce1da None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:25.006990 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da37ea-5db1-4945-ba50-3a31eabce1da None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:25.007301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da37ea-5db1-4945-ba50-3a31eabce1da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:25.008872 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da37ea-5db1-4945-ba50-3a31eabce1da None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:25.024012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584615ec-2a60-4cfa-be57-bdd2cbc1caa6 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:25.028369 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 290/596] 10.222.0.42 () {56 vars in 1139 bytes} [Tue Jul 19 15:17:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:25.029409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da37ea-5db1-4945-ba50-3a31eabce1da None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:25.059477 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:25.063871 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954e2d1-c3c7-4330-91cb-3ba31a8673c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:25.065170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954e2d1-c3c7-4330-91cb-3ba31a8673c0 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:25.065700 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954e2d1-c3c7-4330-91cb-3ba31a8673c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:25.067668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954e2d1-c3c7-4330-91cb-3ba31a8673c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:25.068838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954e2d1-c3c7-4330-91cb-3ba31a8673c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:25.069402 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954e2d1-c3c7-4330-91cb-3ba31a8673c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:25.071037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954e2d1-c3c7-4330-91cb-3ba31a8673c0 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:25.086938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 307/597] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:25.088238 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954e2d1-c3c7-4330-91cb-3ba31a8673c0 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:25.136390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da37ea-5db1-4945-ba50-3a31eabce1da None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:25.172505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61b94e-5106-4bbd-a6d1-fc2bc02a0b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:25.172927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61b94e-5106-4bbd-a6d1-fc2bc02a0b9f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:25.173248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61b94e-5106-4bbd-a6d1-fc2bc02a0b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:25.173528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61b94e-5106-4bbd-a6d1-fc2bc02a0b9f None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:25.174315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61b94e-5106-4bbd-a6d1-fc2bc02a0b9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:25.174638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61b94e-5106-4bbd-a6d1-fc2bc02a0b9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:25.176298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61b94e-5106-4bbd-a6d1-fc2bc02a0b9f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:25.193250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 291/598] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:25.194114 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61b94e-5106-4bbd-a6d1-fc2bc02a0b9f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:25.198645 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:25.214004 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954e2d1-c3c7-4330-91cb-3ba31a8673c0 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:25.252124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0866769-5647-4f92-bfe2-6114378b654e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:25.253917 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0866769-5647-4f92-bfe2-6114378b654e None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:25.254511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0866769-5647-4f92-bfe2-6114378b654e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:25.254993 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0866769-5647-4f92-bfe2-6114378b654e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:25.267274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0866769-5647-4f92-bfe2-6114378b654e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:25.270769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0866769-5647-4f92-bfe2-6114378b654e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:25.275392 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:25.293577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-c0866769-5647-4f92-bfe2-6114378b654e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:25.293577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:25.293577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:25.293577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:25.293577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:25.293577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:25.293577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:25.293577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:25.293577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:25.293577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:25.293577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:25.293577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:25.293577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:25.293577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:25.293577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:25.294505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:25.295406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:25.295406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:25.295406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:25.295406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:25.297738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 308/599] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:25.298717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0866769-5647-4f92-bfe2-6114378b654e None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:25.312470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61b94e-5106-4bbd-a6d1-fc2bc02a0b9f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:25.348397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33655e-1273-4141-86e8-80ef583d3064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:25.349225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33655e-1273-4141-86e8-80ef583d3064 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:25.349554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33655e-1273-4141-86e8-80ef583d3064 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:25.349829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33655e-1273-4141-86e8-80ef583d3064 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:25.357088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33655e-1273-4141-86e8-80ef583d3064 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:25.358592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33655e-1273-4141-86e8-80ef583d3064 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:25.375196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-cb33655e-1273-4141-86e8-80ef583d3064 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:25.375196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:25.375196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:25.375196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:25.375196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:25.375196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:25.375196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:25.375196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:25.375196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:25.375196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:25.375196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:25.375196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:25.375196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:25.375196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:25.375196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:25.376119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:25.377022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:25.377022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:25.377022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:25.377022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:25.377022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:25.377022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:25.377022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:25.377022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:25.377022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 292/600] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:25.382119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33655e-1273-4141-86e8-80ef583d3064 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:25.421365 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0866769-5647-4f92-bfe2-6114378b654e None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:25.470746 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52c003-2d38-4912-a77c-b46a0e88ea24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:25.472707 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52c003-2d38-4912-a77c-b46a0e88ea24 None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:25.473256 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52c003-2d38-4912-a77c-b46a0e88ea24 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:25.474138 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52c003-2d38-4912-a77c-b46a0e88ea24 None admin] PATH_INFO: `/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/215c6f6f1aad486f85ce20590d889cbc/roles/c8578ede4e7041ecaeba79c255cc650b` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:25.490202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:25.496724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52c003-2d38-4912-a77c-b46a0e88ea24 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5f51e47b8e34bf5ac822d8c93853e30, user_id=215c6f6f1aad486f85ce20590d889cbc, role_id=c8578ede4e7041ecaeba79c255cc650b)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:25.497574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33655e-1273-4141-86e8-80ef583d3064 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:25.498710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52c003-2d38-4912-a77c-b46a0e88ea24 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:25.525320 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 309/601] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:17:25 2022] PUT /identity/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/215c6f6f1aad486f85ce20590d889cbc/roles/c8578ede4e7041ecaeba79c255cc650b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:17:25.526603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52c003-2d38-4912-a77c-b46a0e88ea24 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:25.537542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:25.545082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca515206-efc0-4496-966c-ee78a049b0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:25.546146 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca515206-efc0-4496-966c-ee78a049b0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:25.546636 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca515206-efc0-4496-966c-ee78a049b0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:25.547116 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca515206-efc0-4496-966c-ee78a049b0b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:25.548826 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca515206-efc0-4496-966c-ee78a049b0b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:25.549792 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca515206-efc0-4496-966c-ee78a049b0b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:25.551659 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca515206-efc0-4496-966c-ee78a049b0b9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:25.567976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 293/602] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:25.577181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca515206-efc0-4496-966c-ee78a049b0b9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:25.733529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52c003-2d38-4912-a77c-b46a0e88ea24 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:25.756651 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca515206-efc0-4496-966c-ee78a049b0b9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:25.776404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1cd61-d6fa-4958-bdab-7ef940061ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:25.778876 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1cd61-d6fa-4958-bdab-7ef940061ceb None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:25.779662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1cd61-d6fa-4958-bdab-7ef940061ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:25.780109 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1cd61-d6fa-4958-bdab-7ef940061ceb None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:25.781126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1cd61-d6fa-4958-bdab-7ef940061ceb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:25.781591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1cd61-d6fa-4958-bdab-7ef940061ceb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:25.783717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1cd61-d6fa-4958-bdab-7ef940061ceb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:25.796618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 310/603] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:25.803802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1cd61-d6fa-4958-bdab-7ef940061ceb None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:25.813720 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec9547-9a54-4ccf-bbc5-ca3f8dcfadc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:25.814871 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec9547-9a54-4ccf-bbc5-ca3f8dcfadc1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:25.815157 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec9547-9a54-4ccf-bbc5-ca3f8dcfadc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:25.815429 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec9547-9a54-4ccf-bbc5-ca3f8dcfadc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:25.828620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec9547-9a54-4ccf-bbc5-ca3f8dcfadc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:25.837711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec9547-9a54-4ccf-bbc5-ca3f8dcfadc1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:25.852884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-99ec9547-9a54-4ccf-bbc5-ca3f8dcfadc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:25.852884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:25.852884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:25.852884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:25.852884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:25.852884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:25.852884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:25.852884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:25.852884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:25.852884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:25.852884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:25.852884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:25.852884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:25.852884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:25.852884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:25.853883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:25.854770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:25.854770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:25.854770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:25.854770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:25.854770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:25.854770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:25.854770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:25.854770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:25.854770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 294/604] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:25.859877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec9547-9a54-4ccf-bbc5-ca3f8dcfadc1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:25.878862 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:25.949545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:25.986522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1cd61-d6fa-4958-bdab-7ef940061ceb None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:26.022544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669747ac-3326-4efe-82c1-e74dd5dac5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:26.023445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669747ac-3326-4efe-82c1-e74dd5dac5f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:26.023725 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669747ac-3326-4efe-82c1-e74dd5dac5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:26.023996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669747ac-3326-4efe-82c1-e74dd5dac5f2 None admin] PATH_INFO: `/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/7245ef27d2454927a01a7cd56a4c27aa/roles/c8578ede4e7041ecaeba79c255cc650b` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:26.037789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec9547-9a54-4ccf-bbc5-ca3f8dcfadc1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:26.046338 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669747ac-3326-4efe-82c1-e74dd5dac5f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5f51e47b8e34bf5ac822d8c93853e30, user_id=7245ef27d2454927a01a7cd56a4c27aa, role_id=c8578ede4e7041ecaeba79c255cc650b)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:26.047739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669747ac-3326-4efe-82c1-e74dd5dac5f2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:26.067252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 311/605] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:17:25 2022] PUT /identity/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/7245ef27d2454927a01a7cd56a4c27aa/roles/c8578ede4e7041ecaeba79c255cc650b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:17:26.069321 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83911f-7f6f-4c10-97e1-f4d10934a8ff None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:26.069620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83911f-7f6f-4c10-97e1-f4d10934a8ff None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:26.069900 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83911f-7f6f-4c10-97e1-f4d10934a8ff None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:26.075954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 312/606] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:26.078060 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad2667-129b-41a0-846a-372429926d8e None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:26.078324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad2667-129b-41a0-846a-372429926d8e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:26.078601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad2667-129b-41a0-846a-372429926d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:26.080671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e51f9-bdff-446b-a250-48619da9568c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:26.081882 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e51f9-bdff-446b-a250-48619da9568c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:26.082383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e51f9-bdff-446b-a250-48619da9568c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:26.082908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e51f9-bdff-446b-a250-48619da9568c None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:26.083909 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e51f9-bdff-446b-a250-48619da9568c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:26.084511 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e51f9-bdff-446b-a250-48619da9568c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:26.086331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e51f9-bdff-446b-a250-48619da9568c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:26.099736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-26ad2667-129b-41a0-846a-372429926d8e None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:26.101128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-26ad2667-129b-41a0-846a-372429926d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:26.102384 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 295/607] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:26.107720 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e51f9-bdff-446b-a250-48619da9568c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:26.208168 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 313/608] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:26.257231 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad2667-129b-41a0-846a-372429926d8e None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:26.308599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e51f9-bdff-446b-a250-48619da9568c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:26.318847 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:26.327202 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:26.362405 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ba72c-e081-4308-812b-512919f9ad51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:26.364792 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ba72c-e081-4308-812b-512919f9ad51 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:26.365078 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ba72c-e081-4308-812b-512919f9ad51 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:26.365395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ba72c-e081-4308-812b-512919f9ad51 None admin] PATH_INFO: `/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/369bdaded0534142b03ad3cf43a70881/roles/c8578ede4e7041ecaeba79c255cc650b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:26.388806 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ba72c-e081-4308-812b-512919f9ad51 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5f51e47b8e34bf5ac822d8c93853e30, user_id=369bdaded0534142b03ad3cf43a70881, role_id=c8578ede4e7041ecaeba79c255cc650b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:26.390584 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ba72c-e081-4308-812b-512919f9ad51 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:26.412251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 296/609] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:17:26 2022] PUT /identity/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/369bdaded0534142b03ad3cf43a70881/roles/c8578ede4e7041ecaeba79c255cc650b => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:17:26.424789 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad2667-129b-41a0-846a-372429926d8e None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:26.467850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86aa14-65fa-4f3d-b63c-94a48e45bf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:26.468904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86aa14-65fa-4f3d-b63c-94a48e45bf76 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:26.469316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86aa14-65fa-4f3d-b63c-94a48e45bf76 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:26.469736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86aa14-65fa-4f3d-b63c-94a48e45bf76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:26.483335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86aa14-65fa-4f3d-b63c-94a48e45bf76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-1d86aa14-65fa-4f3d-b63c-94a48e45bf76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:26.492776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:26.493984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:26.495016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:26.495016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:26.495016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:26.495016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:26.495016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:26.495016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:26.495016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:26.495016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:26.495016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:26.495016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:26.495016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:26.495016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:26.501252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 314/610] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:26.503364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ba72c-e081-4308-812b-512919f9ad51 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:26.604301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12214d-5fb8-49ce-baba-4dde76772573 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:26.604650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12214d-5fb8-49ce-baba-4dde76772573 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:26.604853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12214d-5fb8-49ce-baba-4dde76772573 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:26.606404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 315/611] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:26.611876 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a91b4-1d13-441c-86ac-19c842c8153b None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:26.612206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a91b4-1d13-441c-86ac-19c842c8153b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:26.613025 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a91b4-1d13-441c-86ac-19c842c8153b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:26.629255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ba72c-e081-4308-812b-512919f9ad51 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:26.662244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-7d9a91b4-1d13-441c-86ac-19c842c8153b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:26.663481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7d9a91b4-1d13-441c-86ac-19c842c8153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:26.683797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a257a-750b-49ac-8b31-08ce88bf523e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:26.684484 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a257a-750b-49ac-8b31-08ce88bf523e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:26.684762 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a257a-750b-49ac-8b31-08ce88bf523e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:26.685036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a257a-750b-49ac-8b31-08ce88bf523e None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:26.685868 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a257a-750b-49ac-8b31-08ce88bf523e None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:26.686170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a257a-750b-49ac-8b31-08ce88bf523e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:26.687540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a257a-750b-49ac-8b31-08ce88bf523e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:26.702619 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 297/612] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:26.708790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a257a-750b-49ac-8b31-08ce88bf523e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:26.740944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:26.751373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 316/613] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:26.775767 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a91b4-1d13-441c-86ac-19c842c8153b None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:26.800886 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:26.822165 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a257a-750b-49ac-8b31-08ce88bf523e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:26.859255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a36e1-1ece-4461-a1d0-f75dc7c977f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:26.860049 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a36e1-1ece-4461-a1d0-f75dc7c977f1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:26.860337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a36e1-1ece-4461-a1d0-f75dc7c977f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:26.860613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a36e1-1ece-4461-a1d0-f75dc7c977f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:26.872769 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a36e1-1ece-4461-a1d0-f75dc7c977f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:26.874233 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a36e1-1ece-4461-a1d0-f75dc7c977f1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:26.893039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-062a36e1-1ece-4461-a1d0-f75dc7c977f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:26.893039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:26.893039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:26.893039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:26.893039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:26.893039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:26.893039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:26.893039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:26.893039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:26.893039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:26.893039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:26.893039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:26.893039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:26.893039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:26.893039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:26.894564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:26.895452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:26.895452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:26.895452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:26.895452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:26.909526 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 298/614] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:26.909526 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a36e1-1ece-4461-a1d0-f75dc7c977f1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:26.937576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a91b4-1d13-441c-86ac-19c842c8153b None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:26.998108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:27.007456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2fe4a-a15a-414a-a549-877e274ad6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:27.008622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2fe4a-a15a-414a-a549-877e274ad6bf None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:27.009111 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2fe4a-a15a-414a-a549-877e274ad6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:27.009622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2fe4a-a15a-414a-a549-877e274ad6bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:27.025249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a36e1-1ece-4461-a1d0-f75dc7c977f1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:27.031278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2fe4a-a15a-414a-a549-877e274ad6bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-03e2fe4a-a15a-414a-a549-877e274ad6bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:27.047562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:27.048549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:27.049441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:27.049441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:27.049441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:27.049441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:27.049441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:27.049441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:27.049441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:27.049441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:27.049441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:27.049441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:27.049441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:27.049441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:27.052287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 317/615] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:27.057802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62031f18-cb66-42b3-807e-8a0b7cd0de29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:27.058443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2fe4a-a15a-414a-a549-877e274ad6bf None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:27.058554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62031f18-cb66-42b3-807e-8a0b7cd0de29 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:27.058744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62031f18-cb66-42b3-807e-8a0b7cd0de29 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:27.059182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62031f18-cb66-42b3-807e-8a0b7cd0de29 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:27.060031 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62031f18-cb66-42b3-807e-8a0b7cd0de29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:27.060346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62031f18-cb66-42b3-807e-8a0b7cd0de29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:27.061754 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62031f18-cb66-42b3-807e-8a0b7cd0de29 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:27.080083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 299/616] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:27.082323 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62031f18-cb66-42b3-807e-8a0b7cd0de29 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:27.219948 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:27.255711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2fe4a-a15a-414a-a549-877e274ad6bf None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:27.275322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62031f18-cb66-42b3-807e-8a0b7cd0de29 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:27.313441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f7957-1f8f-4e0a-aa1c-02f8e538350d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:27.315255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f7957-1f8f-4e0a-aa1c-02f8e538350d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:27.316290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f7957-1f8f-4e0a-aa1c-02f8e538350d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:27.316779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f7957-1f8f-4e0a-aa1c-02f8e538350d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:27.324499 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f7957-1f8f-4e0a-aa1c-02f8e538350d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:27.325432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:27.329642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f7957-1f8f-4e0a-aa1c-02f8e538350d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-6c9f7957-1f8f-4e0a-aa1c-02f8e538350d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:27.346911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:27.348531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:27.349546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:27.349546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:27.349546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:27.349546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:27.349546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:27.349546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:27.349546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:27.349546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:27.349546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:17:27.349546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:27.356550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 300/617] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Jul 19 15:17:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:27.357953 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f7957-1f8f-4e0a-aa1c-02f8e538350d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:27.373853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf954e82-fe11-43ce-ba94-0c2430e3f10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:27.374544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf954e82-fe11-43ce-ba94-0c2430e3f10f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:27.374837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf954e82-fe11-43ce-ba94-0c2430e3f10f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:27.375118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf954e82-fe11-43ce-ba94-0c2430e3f10f None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:27.375912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf954e82-fe11-43ce-ba94-0c2430e3f10f None admin] RBAC: Adding query filter params (name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:27.376205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf954e82-fe11-43ce-ba94-0c2430e3f10f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:27.377674 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf954e82-fe11-43ce-ba94-0c2430e3f10f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:27.407437 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 318/618] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:27.412716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf954e82-fe11-43ce-ba94-0c2430e3f10f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:27.474733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f7957-1f8f-4e0a-aa1c-02f8e538350d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:27.522504 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce3029-441f-454b-b1cb-6ffffcc883f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:27.524106 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce3029-441f-454b-b1cb-6ffffcc883f2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:27.524968 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce3029-441f-454b-b1cb-6ffffcc883f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:27.525491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce3029-441f-454b-b1cb-6ffffcc883f2 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:27.526824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce3029-441f-454b-b1cb-6ffffcc883f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:27.527436 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce3029-441f-454b-b1cb-6ffffcc883f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:27.529129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce3029-441f-454b-b1cb-6ffffcc883f2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:27.559329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 301/619] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Jul 19 15:17:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:27.561846 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce3029-441f-454b-b1cb-6ffffcc883f2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:27.618269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf954e82-fe11-43ce-ba94-0c2430e3f10f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:27.665583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:27.686386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23748c-6618-49e1-b000-e8c6051e3b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:27.687205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23748c-6618-49e1-b000-e8c6051e3b82 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:27.687474 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23748c-6618-49e1-b000-e8c6051e3b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:27.687754 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23748c-6618-49e1-b000-e8c6051e3b82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:27.699220 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce3029-441f-454b-b1cb-6ffffcc883f2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:27.712835 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23748c-6618-49e1-b000-e8c6051e3b82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:27.714328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23748c-6618-49e1-b000-e8c6051e3b82 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:27.723421 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:27.740240 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b6b3e-f087-4492-88b2-25fbf143f12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:27.741122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b6b3e-f087-4492-88b2-25fbf143f12b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:27.741683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b6b3e-f087-4492-88b2-25fbf143f12b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:27.742139 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b6b3e-f087-4492-88b2-25fbf143f12b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:27.747999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-9e23748c-6618-49e1-b000-e8c6051e3b82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:27.747999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:27.747999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:27.747999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:27.747999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:27.747999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:27.747999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:27.747999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:27.747999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:27.747999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:27.747999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:27.747999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:27.747999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:27.747999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:27.747999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:27.749064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:27.750181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:27.750181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:27.750181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:27.750181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:27.757649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 319/620] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:27.762780 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23748c-6618-49e1-b000-e8c6051e3b82 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:27.825125 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b6b3e-f087-4492-88b2-25fbf143f12b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:27.826915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b6b3e-f087-4492-88b2-25fbf143f12b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:27.889618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:27.953329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 302/621] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 15:17:27.955569 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b6b3e-f087-4492-88b2-25fbf143f12b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:28.071944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b6b3e-f087-4492-88b2-25fbf143f12b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:28.079042 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23748c-6618-49e1-b000-e8c6051e3b82 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:28.111473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb323df-5c5a-4c25-bcde-1e862953a2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:28.113838 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb323df-5c5a-4c25-bcde-1e862953a2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:28.114296 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb323df-5c5a-4c25-bcde-1e862953a2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:28.115008 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb323df-5c5a-4c25-bcde-1e862953a2d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:28.126718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb323df-5c5a-4c25-bcde-1e862953a2d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:28.129471 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb323df-5c5a-4c25-bcde-1e862953a2d1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:28.151198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3bb323df-5c5a-4c25-bcde-1e862953a2d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:28.151198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:28.151198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:28.151198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:28.151198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:28.151198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:28.151198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:28.151198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:28.151198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:28.151198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:28.151198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:28.151198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:28.151198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:28.151198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:28.151198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:28.152363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:28.153568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:28.153568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:28.153568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:28.153568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:28.160673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 303/622] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:28.162295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb323df-5c5a-4c25-bcde-1e862953a2d1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:28.197540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:28.213864 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc99a21-8846-4a87-8903-43d7e41d2dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:28.214551 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc99a21-8846-4a87-8903-43d7e41d2dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:28.214833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc99a21-8846-4a87-8903-43d7e41d2dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:28.215107 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc99a21-8846-4a87-8903-43d7e41d2dc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:28.215922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc99a21-8846-4a87-8903-43d7e41d2dc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:28.216142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc99a21-8846-4a87-8903-43d7e41d2dc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:28.217770 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc99a21-8846-4a87-8903-43d7e41d2dc2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:28.255923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 320/623] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:28.262169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc99a21-8846-4a87-8903-43d7e41d2dc2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:28.290814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb323df-5c5a-4c25-bcde-1e862953a2d1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:28.335768 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab6b9e-d8e7-4059-bf7d-c81b4c5fc418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:28.336965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab6b9e-d8e7-4059-bf7d-c81b4c5fc418 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:28.337285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab6b9e-d8e7-4059-bf7d-c81b4c5fc418 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:28.337567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab6b9e-d8e7-4059-bf7d-c81b4c5fc418 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:28.338307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab6b9e-d8e7-4059-bf7d-c81b4c5fc418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:28.338618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab6b9e-d8e7-4059-bf7d-c81b4c5fc418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:28.339993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab6b9e-d8e7-4059-bf7d-c81b4c5fc418 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:28.361239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 304/624] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:28.362688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e94373-d0f3-4baa-8477-8c983a760dd1 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:28.362688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e94373-d0f3-4baa-8477-8c983a760dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:28.363447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e94373-d0f3-4baa-8477-8c983a760dd1 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:28.364961 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 305/625] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:28.367096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e94373-d0f3-4baa-8477-8c983a760dd1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:28.387993 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:28.439998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc99a21-8846-4a87-8903-43d7e41d2dc2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:28.453565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:28.477406 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e94373-d0f3-4baa-8477-8c983a760dd1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:28.480217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed554b7-2112-4994-a4a2-619cbc3eef5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:28.481497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed554b7-2112-4994-a4a2-619cbc3eef5d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:28.482000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed554b7-2112-4994-a4a2-619cbc3eef5d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:28.482487 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed554b7-2112-4994-a4a2-619cbc3eef5d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:28.491283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed554b7-2112-4994-a4a2-619cbc3eef5d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:28.493070 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed554b7-2112-4994-a4a2-619cbc3eef5d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-aed554b7-2112-4994-a4a2-619cbc3eef5d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:28.510087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:28.511091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:28.512053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:28.512053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:28.512053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:28.512053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:28.512053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:28.512053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:28.512053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:28.512053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:28.512053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 15:17:28.512053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:28.514773 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6284a9a-696f-4510-a69e-46a43b7c0311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:28.515276 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 321/626] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:17:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:28.515585 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6284a9a-696f-4510-a69e-46a43b7c0311 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:28.515870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6284a9a-696f-4510-a69e-46a43b7c0311 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:28.517379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea6957-894e-4856-a6d1-7ccd54ed373c None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:28.517951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea6957-894e-4856-a6d1-7ccd54ed373c None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:28.518475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea6957-894e-4856-a6d1-7ccd54ed373c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:28.521525 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6284a9a-696f-4510-a69e-46a43b7c0311 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:28.534429 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6284a9a-696f-4510-a69e-46a43b7c0311 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:28.536911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6284a9a-696f-4510-a69e-46a43b7c0311 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:28.541797 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-0cea6957-894e-4856-a6d1-7ccd54ed373c None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:28.542979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0cea6957-894e-4856-a6d1-7ccd54ed373c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:28.552420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e6284a9a-696f-4510-a69e-46a43b7c0311 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:28.552420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:28.552420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:28.552420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:28.552420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:28.552420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:28.552420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:28.552420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:28.552420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:28.552420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:28.552420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:28.552420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:28.552420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:28.552420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:28.552420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:28.553474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:28.554472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:28.554472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:28.554472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:28.554472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:28.554472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:28.554472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:28.554472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:28.554472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:28.554472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 306/627] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:28.554976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6284a9a-696f-4510-a69e-46a43b7c0311 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:28.616817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 322/628] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:28.618422 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc354115-76eb-4579-add1-4ea6024c09af None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:28.618708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc354115-76eb-4579-add1-4ea6024c09af None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:28.618980 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc354115-76eb-4579-add1-4ea6024c09af None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:28.620384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 323/629] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:28.620910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc354115-76eb-4579-add1-4ea6024c09af None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:28.673710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:28.689652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6284a9a-696f-4510-a69e-46a43b7c0311 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:28.718957 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:28.729554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dcd77-a41a-4c86-b56e-829bbf8a020d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:28.730473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dcd77-a41a-4c86-b56e-829bbf8a020d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:28.730473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dcd77-a41a-4c86-b56e-829bbf8a020d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:28.730759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dcd77-a41a-4c86-b56e-829bbf8a020d None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:28.731607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dcd77-a41a-4c86-b56e-829bbf8a020d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:28.731921 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dcd77-a41a-4c86-b56e-829bbf8a020d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:28.733352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dcd77-a41a-4c86-b56e-829bbf8a020d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:28.741021 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc354115-76eb-4579-add1-4ea6024c09af None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:28.751715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 307/630] 10.222.0.42 () {56 vars in 1135 bytes} [Tue Jul 19 15:17:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:28.752974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed47ed-32ea-4aa9-b8b4-31f08f847824 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:28.753309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed47ed-32ea-4aa9-b8b4-31f08f847824 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:28.753582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed47ed-32ea-4aa9-b8b4-31f08f847824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:28.770415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a1ed47ed-32ea-4aa9-b8b4-31f08f847824 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:28.771593 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a1ed47ed-32ea-4aa9-b8b4-31f08f847824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:28.783448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85963e54-9c80-4427-9fae-dce960a883b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:28.784826 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85963e54-9c80-4427-9fae-dce960a883b6 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:28.785076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85963e54-9c80-4427-9fae-dce960a883b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:28.785400 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85963e54-9c80-4427-9fae-dce960a883b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:28.786255 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85963e54-9c80-4427-9fae-dce960a883b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:28.786567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85963e54-9c80-4427-9fae-dce960a883b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:28.788176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85963e54-9c80-4427-9fae-dce960a883b6 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:28.806176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 324/631] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:28.806176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85963e54-9c80-4427-9fae-dce960a883b6 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:28.846413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 308/632] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:28.847808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed47ed-32ea-4aa9-b8b4-31f08f847824 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:28.915006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85963e54-9c80-4427-9fae-dce960a883b6 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:28.917297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:28.948512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed47ed-32ea-4aa9-b8b4-31f08f847824 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:28.949047 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c4020-671d-4543-8fca-20a0dc005020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:28.949871 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c4020-671d-4543-8fca-20a0dc005020 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:28.950150 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c4020-671d-4543-8fca-20a0dc005020 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:28.950420 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c4020-671d-4543-8fca-20a0dc005020 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:28.960948 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c4020-671d-4543-8fca-20a0dc005020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:28.963167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-914c4020-671d-4543-8fca-20a0dc005020 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:28.968685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:28.969897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:28.970974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:28.970974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:28.970974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:28.970974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:28.970974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:28.970974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:28.970974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:28.970974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:28.970974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:28.970974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:28.970974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:28.970974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:28.970974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 325/633] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:28.970974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c4020-671d-4543-8fca-20a0dc005020 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:28.989475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56ffb5-12a6-4316-9eff-68d14008da62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:28.990266 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56ffb5-12a6-4316-9eff-68d14008da62 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:28.990555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56ffb5-12a6-4316-9eff-68d14008da62 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:28.990830 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56ffb5-12a6-4316-9eff-68d14008da62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:29.000730 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56ffb5-12a6-4316-9eff-68d14008da62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:29.002207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56ffb5-12a6-4316-9eff-68d14008da62 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:29.020169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3d56ffb5-12a6-4316-9eff-68d14008da62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:29.020169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:29.020169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:29.020169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:29.020169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:29.020169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:29.020169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:29.020169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:29.020169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:29.020169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:29.020169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:29.020169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:29.020169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:29.020169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:29.020169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:29.021304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:29.022223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:29.022223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:29.022223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:29.022223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:29.022223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 309/634] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:29.022828 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c3f19-a657-4c70-9dd3-3b560ddc3aae None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:29.023112 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c3f19-a657-4c70-9dd3-3b560ddc3aae None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:29.023389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c3f19-a657-4c70-9dd3-3b560ddc3aae None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:29.024640 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 310/635] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:29.025360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c3f19-a657-4c70-9dd3-3b560ddc3aae None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:29.079938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c4020-671d-4543-8fca-20a0dc005020 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:29.114587 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79ae59-b352-4ddc-807a-4ab02eb4900c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:29.115246 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79ae59-b352-4ddc-807a-4ab02eb4900c None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:29.115531 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79ae59-b352-4ddc-807a-4ab02eb4900c None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:29.115811 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79ae59-b352-4ddc-807a-4ab02eb4900c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:29.116649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79ae59-b352-4ddc-807a-4ab02eb4900c None admin] RBAC: Adding query filter params (role.id=c8578ede4e7041ecaeba79c255cc650b, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=bff6a2d4bc734904b2615608a86b097d) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:29.116964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79ae59-b352-4ddc-807a-4ab02eb4900c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:29.118465 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79ae59-b352-4ddc-807a-4ab02eb4900c None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:29.124169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 326/636] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:28 2022] GET /identity/v3/role_assignments?user.id=bff6a2d4bc734904b2615608a86b097d&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=c8578ede4e7041ecaeba79c255cc650b => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:29.124974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79ae59-b352-4ddc-807a-4ab02eb4900c None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:29.126475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c3f19-a657-4c70-9dd3-3b560ddc3aae None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:29.157882 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:29.169675 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90bf30-bfcb-4488-89a5-c288e13cbb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:29.170485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90bf30-bfcb-4488-89a5-c288e13cbb31 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:29.170771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90bf30-bfcb-4488-89a5-c288e13cbb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:29.171051 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90bf30-bfcb-4488-89a5-c288e13cbb31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:29.184482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90bf30-bfcb-4488-89a5-c288e13cbb31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-6b90bf30-bfcb-4488-89a5-c288e13cbb31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:29.192573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:29.193685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:29.194541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:29.194541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:29.194541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:29.194541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:29.194541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:29.194541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:29.194541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:29.194541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:29.194541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:29.194541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:29.194541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:29.194541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:29.194541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 311/637] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:29.196067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90bf30-bfcb-4488-89a5-c288e13cbb31 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:29.227121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:29.259029 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79ae59-b352-4ddc-807a-4ab02eb4900c None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:29.302982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fb7a8-9fa4-43bb-84b9-e56ca8b9ce69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:29.303659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fb7a8-9fa4-43bb-84b9-e56ca8b9ce69 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:29.303949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fb7a8-9fa4-43bb-84b9-e56ca8b9ce69 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:29.304233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fb7a8-9fa4-43bb-84b9-e56ca8b9ce69 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:29.305490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fb7a8-9fa4-43bb-84b9-e56ca8b9ce69 None admin] RBAC: Adding query filter params (name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:29.306262 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fb7a8-9fa4-43bb-84b9-e56ca8b9ce69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:29.307250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fb7a8-9fa4-43bb-84b9-e56ca8b9ce69 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:29.317108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 327/638] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:29.318395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40893881-ff2d-4d79-8bdb-95adf7c425f2 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:29.318685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40893881-ff2d-4d79-8bdb-95adf7c425f2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:29.318964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40893881-ff2d-4d79-8bdb-95adf7c425f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:29.319845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90bf30-bfcb-4488-89a5-c288e13cbb31 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:29.336316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-40893881-ff2d-4d79-8bdb-95adf7c425f2 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:29.337586 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-40893881-ff2d-4d79-8bdb-95adf7c425f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:29.364271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b4485-0fee-4887-ab7b-189eaa6bc9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:29.365840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b4485-0fee-4887-ab7b-189eaa6bc9ce None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:29.366139 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b4485-0fee-4887-ab7b-189eaa6bc9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:29.366697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b4485-0fee-4887-ab7b-189eaa6bc9ce None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:29.367446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b4485-0fee-4887-ab7b-189eaa6bc9ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:29.367921 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b4485-0fee-4887-ab7b-189eaa6bc9ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:29.369709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b4485-0fee-4887-ab7b-189eaa6bc9ce None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:29.386813 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 312/639] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:29.388237 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b4485-0fee-4887-ab7b-189eaa6bc9ce None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:29.414585 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 328/640] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:29.415438 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40893881-ff2d-4d79-8bdb-95adf7c425f2 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:29.445242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:29.488927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:29.505517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b4485-0fee-4887-ab7b-189eaa6bc9ce None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:29.532804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40893881-ff2d-4d79-8bdb-95adf7c425f2 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:29.541309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af234964-3a23-490c-b425-4a582b8f915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:29.541993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af234964-3a23-490c-b425-4a582b8f915e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:29.542279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af234964-3a23-490c-b425-4a582b8f915e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:29.542564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af234964-3a23-490c-b425-4a582b8f915e None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:29.543320 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af234964-3a23-490c-b425-4a582b8f915e None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:29.543631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af234964-3a23-490c-b425-4a582b8f915e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:29.544998 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af234964-3a23-490c-b425-4a582b8f915e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:29.554153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 313/641] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:29.558507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af234964-3a23-490c-b425-4a582b8f915e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:29.569076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7bae4-e276-42ac-9d95-ab75fdba4921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:29.570107 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7bae4-e276-42ac-9d95-ab75fdba4921 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:29.570241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7bae4-e276-42ac-9d95-ab75fdba4921 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:29.570449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7bae4-e276-42ac-9d95-ab75fdba4921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:29.580847 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7bae4-e276-42ac-9d95-ab75fdba4921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:29.583015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7bae4-e276-42ac-9d95-ab75fdba4921 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:29.601435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-4cd7bae4-e276-42ac-9d95-ab75fdba4921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:29.601435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:29.601435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:29.601435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:29.601435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:29.601435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:29.601435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:29.601435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:29.601435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:29.601435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:29.601435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:29.601435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:29.601435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:29.601435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:29.601435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:29.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:29.603248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:29.603248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:29.603248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:29.603248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:29.607263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 329/642] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:29.608310 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7bae4-e276-42ac-9d95-ab75fdba4921 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:29.670162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af234964-3a23-490c-b425-4a582b8f915e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:29.685484 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:29.708460 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf4c8e-3065-4a6e-8edf-707b37ef09fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:29.709282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf4c8e-3065-4a6e-8edf-707b37ef09fa None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:29.709569 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf4c8e-3065-4a6e-8edf-707b37ef09fa None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:29.709844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf4c8e-3065-4a6e-8edf-707b37ef09fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:29.716761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf4c8e-3065-4a6e-8edf-707b37ef09fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:29.718327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf4c8e-3065-4a6e-8edf-707b37ef09fa None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:29.721485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7bae4-e276-42ac-9d95-ab75fdba4921 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:29.733153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-bbcf4c8e-3065-4a6e-8edf-707b37ef09fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:29.733153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:29.733153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:29.733153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:29.733153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:29.733153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:29.733153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:29.733153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:29.733153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:29.733153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:29.733153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:29.733153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:29.733153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:29.733153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:29.733153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:29.734136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:29.735088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:29.735088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:29.735088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:29.735088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:29.735088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:29.735088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:29.735088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:29.735088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:29.735088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 314/643] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:29.735591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf4c8e-3065-4a6e-8edf-707b37ef09fa None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:29.747145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:29.764056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed88ca-9411-4b9a-9f33-8675d38574d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:29.764854 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed88ca-9411-4b9a-9f33-8675d38574d5 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:29.765168 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed88ca-9411-4b9a-9f33-8675d38574d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:29.765425 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed88ca-9411-4b9a-9f33-8675d38574d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:29.776424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed88ca-9411-4b9a-9f33-8675d38574d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-77ed88ca-9411-4b9a-9f33-8675d38574d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:29.785367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:29.786330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:29.787221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:29.787221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:29.787221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:29.787221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:29.787221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:29.787221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:29.787221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:29.787221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:29.787221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:29.787221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:29.787221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:29.787221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:29.787221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 330/644] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:29.788060 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed88ca-9411-4b9a-9f33-8675d38574d5 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:29.850481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf4c8e-3065-4a6e-8edf-707b37ef09fa None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:29.887764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b234f-77d5-439a-866f-0150ef5b1838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:29.888685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b234f-77d5-439a-866f-0150ef5b1838 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:29.889038 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b234f-77d5-439a-866f-0150ef5b1838 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:29.889405 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b234f-77d5-439a-866f-0150ef5b1838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:29.893258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed88ca-9411-4b9a-9f33-8675d38574d5 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:29.900295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b234f-77d5-439a-866f-0150ef5b1838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:29.902115 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b234f-77d5-439a-866f-0150ef5b1838 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:29.921228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-af9b234f-77d5-439a-866f-0150ef5b1838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:29.921228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:29.921228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:29.921228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:29.921228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:29.921228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:29.921228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:29.921228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:29.921228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:29.921228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:29.921228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:29.921228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:29.921228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:29.921228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:29.921228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:29.922301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:29.923494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:29.923494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:29.923494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:29.923494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:29.925686 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:29.929696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 315/645] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:29.931572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b234f-77d5-439a-866f-0150ef5b1838 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:29.931752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603c427-ae76-4f28-9e97-22dfa60951f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:29.932421 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603c427-ae76-4f28-9e97-22dfa60951f7 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:29.932708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603c427-ae76-4f28-9e97-22dfa60951f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:29.932989 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603c427-ae76-4f28-9e97-22dfa60951f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:29.933756 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603c427-ae76-4f28-9e97-22dfa60951f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:29.934066 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603c427-ae76-4f28-9e97-22dfa60951f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:29.935455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603c427-ae76-4f28-9e97-22dfa60951f7 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:29.949849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 331/646] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:29.954960 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603c427-ae76-4f28-9e97-22dfa60951f7 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:30.001835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:30.045602 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b234f-77d5-439a-866f-0150ef5b1838 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:30.062027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603c427-ae76-4f28-9e97-22dfa60951f7 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:30.084268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d8761-95c0-44d6-be80-fb7d7a5b5b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:30.084934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d8761-95c0-44d6-be80-fb7d7a5b5b14 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:30.085249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d8761-95c0-44d6-be80-fb7d7a5b5b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:30.085527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d8761-95c0-44d6-be80-fb7d7a5b5b14 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:30.086322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d8761-95c0-44d6-be80-fb7d7a5b5b14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:30.086632 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d8761-95c0-44d6-be80-fb7d7a5b5b14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:30.088060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d8761-95c0-44d6-be80-fb7d7a5b5b14 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:30.097661 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141551fd-6959-4f97-a8b1-089bc33d0b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:30.098372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141551fd-6959-4f97-a8b1-089bc33d0b21 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:30.098660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141551fd-6959-4f97-a8b1-089bc33d0b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:30.098943 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141551fd-6959-4f97-a8b1-089bc33d0b21 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:30.099697 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141551fd-6959-4f97-a8b1-089bc33d0b21 None admin] RBAC: Adding query filter params (name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:30.104930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 316/647] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:30.105082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141551fd-6959-4f97-a8b1-089bc33d0b21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:30.105197 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d8761-95c0-44d6-be80-fb7d7a5b5b14 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:30.105301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141551fd-6959-4f97-a8b1-089bc33d0b21 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:30.113209 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 332/648] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:30.113907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141551fd-6959-4f97-a8b1-089bc33d0b21 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:30.170501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:30.211347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d8761-95c0-44d6-be80-fb7d7a5b5b14 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:30.222986 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141551fd-6959-4f97-a8b1-089bc33d0b21 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:30.233333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:30.251386 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374bfe98-0d34-4936-99fa-4fbc4a180a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:30.252592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374bfe98-0d34-4936-99fa-4fbc4a180a91 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:30.253342 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374bfe98-0d34-4936-99fa-4fbc4a180a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:30.253672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374bfe98-0d34-4936-99fa-4fbc4a180a91 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:30.254557 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374bfe98-0d34-4936-99fa-4fbc4a180a91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:30.254726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374bfe98-0d34-4936-99fa-4fbc4a180a91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:30.256064 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374bfe98-0d34-4936-99fa-4fbc4a180a91 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:30.267001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437bf04-35d7-4bac-930c-71dd7b8f1084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:30.267787 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437bf04-35d7-4bac-930c-71dd7b8f1084 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:30.268068 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437bf04-35d7-4bac-930c-71dd7b8f1084 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:30.268340 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437bf04-35d7-4bac-930c-71dd7b8f1084 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:30.272993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 317/649] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:30.273929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374bfe98-0d34-4936-99fa-4fbc4a180a91 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:30.278770 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9437bf04-35d7-4bac-930c-71dd7b8f1084 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:30.280232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9437bf04-35d7-4bac-930c-71dd7b8f1084 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-9437bf04-35d7-4bac-930c-71dd7b8f1084 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:30.295849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:30.296863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:30.304297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:30.304297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:30.304297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:30.304297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:30.304297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:30.304297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:30.304297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:30.304297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:30.304297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 19 15:17:30.304297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:30.304297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 333/650] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:30.304297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437bf04-35d7-4bac-930c-71dd7b8f1084 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:30.401302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374bfe98-0d34-4936-99fa-4fbc4a180a91 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:30.411408 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437bf04-35d7-4bac-930c-71dd7b8f1084 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:30.426575 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:30.436787 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e965a-eead-4233-96e6-87ef53085a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:30.437715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e965a-eead-4233-96e6-87ef53085a5d None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:30.437994 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e965a-eead-4233-96e6-87ef53085a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:30.438271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e965a-eead-4233-96e6-87ef53085a5d None admin] PATH_INFO: `/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/96faaac55bc5408685c68a6177d91971/roles/c8578ede4e7041ecaeba79c255cc650b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:30.451092 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf88a9-0f1e-47ab-ad6a-19f53bc87daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:30.451886 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf88a9-0f1e-47ab-ad6a-19f53bc87daf None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:30.452295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf88a9-0f1e-47ab-ad6a-19f53bc87daf None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:30.452677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf88a9-0f1e-47ab-ad6a-19f53bc87daf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:30.453337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e965a-eead-4233-96e6-87ef53085a5d None admin] RBAC: Authorizing `identity:create_grant(project_id=a5f51e47b8e34bf5ac822d8c93853e30, user_id=96faaac55bc5408685c68a6177d91971, role_id=c8578ede4e7041ecaeba79c255cc650b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:30.454778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e965a-eead-4233-96e6-87ef53085a5d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:30.464439 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf88a9-0f1e-47ab-ad6a-19f53bc87daf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:30.466015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf88a9-0f1e-47ab-ad6a-19f53bc87daf None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:30.468880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 318/651] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:17:30 2022] PUT /identity/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/96faaac55bc5408685c68a6177d91971/roles/c8578ede4e7041ecaeba79c255cc650b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:17:30.469728 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e965a-eead-4233-96e6-87ef53085a5d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:30.494449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-d5cf88a9-0f1e-47ab-ad6a-19f53bc87daf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:30.494449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:30.494449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:30.494449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:30.494449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:30.494449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:30.494449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:30.494449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:30.494449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:30.494449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:30.494449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:30.494449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:30.494449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:30.494449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:30.494449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:30.495456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:30.496366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:30.496366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:30.496366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:30.496366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:30.498801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 334/652] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:30.499920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf88a9-0f1e-47ab-ad6a-19f53bc87daf None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:30.512592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:30.597332 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e965a-eead-4233-96e6-87ef53085a5d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:30.618290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf88a9-0f1e-47ab-ad6a-19f53bc87daf None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:30.638832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54a130-8e8d-45dd-8547-84fbd095025e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:30.639640 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54a130-8e8d-45dd-8547-84fbd095025e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:30.639929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54a130-8e8d-45dd-8547-84fbd095025e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:30.640204 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54a130-8e8d-45dd-8547-84fbd095025e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:30.648003 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54a130-8e8d-45dd-8547-84fbd095025e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:30.649475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54a130-8e8d-45dd-8547-84fbd095025e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:30.655985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ce90b-e3a5-4c21-9704-86be09d75bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:30.656880 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ce90b-e3a5-4c21-9704-86be09d75bee None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:30.657224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ce90b-e3a5-4c21-9704-86be09d75bee None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:30.657603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ce90b-e3a5-4c21-9704-86be09d75bee None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:30.658562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ce90b-e3a5-4c21-9704-86be09d75bee None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:30.658980 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ce90b-e3a5-4c21-9704-86be09d75bee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:30.661249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ce90b-e3a5-4c21-9704-86be09d75bee None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-8b54a130-8e8d-45dd-8547-84fbd095025e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:30.666541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:30.667790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:30.669096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:30.669096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:30.669096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:30.669096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:30.669096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:30.669096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:30.669096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:30.669096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:30.669096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 15:17:30.669096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:30.669096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 319/653] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:30.669096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54a130-8e8d-45dd-8547-84fbd095025e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:30.690071 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 335/654] 10.222.0.42 () {56 vars in 1137 bytes} [Tue Jul 19 15:17:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:30.690071 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ce90b-e3a5-4c21-9704-86be09d75bee None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:30.717577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:30.795927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:30.802765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ce90b-e3a5-4c21-9704-86be09d75bee None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:30.806934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54a130-8e8d-45dd-8547-84fbd095025e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:30.839762 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc8f2c-7044-4fb9-9fe4-d6bf1642948d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:30.840713 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc8f2c-7044-4fb9-9fe4-d6bf1642948d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:30.841153 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc8f2c-7044-4fb9-9fe4-d6bf1642948d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:30.841535 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc8f2c-7044-4fb9-9fe4-d6bf1642948d None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:30.842526 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc8f2c-7044-4fb9-9fe4-d6bf1642948d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:30.842939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc8f2c-7044-4fb9-9fe4-d6bf1642948d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:30.844470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc8f2c-7044-4fb9-9fe4-d6bf1642948d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:30.844780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78648270-5970-4245-b075-bca80a5ed8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:30.845515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78648270-5970-4245-b075-bca80a5ed8ee None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:30.845798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78648270-5970-4245-b075-bca80a5ed8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:30.846075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78648270-5970-4245-b075-bca80a5ed8ee None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:30.847595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78648270-5970-4245-b075-bca80a5ed8ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:30.847909 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78648270-5970-4245-b075-bca80a5ed8ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:30.849417 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78648270-5970-4245-b075-bca80a5ed8ee None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:30.861825 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 320/655] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:30.862528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78648270-5970-4245-b075-bca80a5ed8ee None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:30.869428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 336/656] 10.222.0.42 () {56 vars in 1137 bytes} [Tue Jul 19 15:17:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:30.870538 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc8f2c-7044-4fb9-9fe4-d6bf1642948d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:30.966068 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:30.978950 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78648270-5970-4245-b075-bca80a5ed8ee None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:30.987396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc8f2c-7044-4fb9-9fe4-d6bf1642948d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:31.017322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c35c6-8968-46ec-be93-108cb11c0c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:31.018232 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c35c6-8968-46ec-be93-108cb11c0c11 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:31.019485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c35c6-8968-46ec-be93-108cb11c0c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:31.019978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c35c6-8968-46ec-be93-108cb11c0c11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:31.025296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e1cc3-c3f0-43bb-8eca-0a1c58e05be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:31.026402 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e1cc3-c3f0-43bb-8eca-0a1c58e05be6 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:31.026694 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e1cc3-c3f0-43bb-8eca-0a1c58e05be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:31.026974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e1cc3-c3f0-43bb-8eca-0a1c58e05be6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:31.035345 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e1cc3-c3f0-43bb-8eca-0a1c58e05be6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:31.037102 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e1cc3-c3f0-43bb-8eca-0a1c58e05be6 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:31.041554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-0b9e1cc3-c3f0-43bb-8eca-0a1c58e05be6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:31.055148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:31.056333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:31.057514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:31.057514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:31.057514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:31.057514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:31.057514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:31.057514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:31.057514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:31.057514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:31.057514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 15:17:31.057514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:31.060975 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 337/657] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:31.061838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e1cc3-c3f0-43bb-8eca-0a1c58e05be6 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:31.103446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c35c6-8968-46ec-be93-108cb11c0c11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:31.104931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c35c6-8968-46ec-be93-108cb11c0c11 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:31.169597 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e1cc3-c3f0-43bb-8eca-0a1c58e05be6 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:31.207191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd2cf1-51db-4997-bbf0-217f2ba09075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:31.208274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd2cf1-51db-4997-bbf0-217f2ba09075 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:31.208809 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd2cf1-51db-4997-bbf0-217f2ba09075 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:31.209351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd2cf1-51db-4997-bbf0-217f2ba09075 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:31.214208 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 321/658] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:17:31.215186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c35c6-8968-46ec-be93-108cb11c0c11 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:31.223531 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:31.276854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:31.291819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd2cf1-51db-4997-bbf0-217f2ba09075 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:31.293313 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd2cf1-51db-4997-bbf0-217f2ba09075 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:31.332246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c35c6-8968-46ec-be93-108cb11c0c11 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:31.368741 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc50be-2e9c-40d3-ad80-b9085336c74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:31.369442 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc50be-2e9c-40d3-ad80-b9085336c74f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:31.369729 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc50be-2e9c-40d3-ad80-b9085336c74f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:31.370314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc50be-2e9c-40d3-ad80-b9085336c74f None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:31.370839 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc50be-2e9c-40d3-ad80-b9085336c74f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:31.371152 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc50be-2e9c-40d3-ad80-b9085336c74f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:31.372609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc50be-2e9c-40d3-ad80-b9085336c74f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:31.395398 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 322/659] 10.222.0.42 () {56 vars in 1139 bytes} [Tue Jul 19 15:17:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:31.395398 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc50be-2e9c-40d3-ad80-b9085336c74f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:31.407667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 338/660] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:17:31.412712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa46af1-b612-4da5-b7db-cdb169707c7a None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:31.412712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa46af1-b612-4da5-b7db-cdb169707c7a None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:31.412712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa46af1-b612-4da5-b7db-cdb169707c7a None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:31.413188 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 339/661] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:31.414275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa46af1-b612-4da5-b7db-cdb169707c7a None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:31.460840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:31.499840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc50be-2e9c-40d3-ad80-b9085336c74f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:31.509459 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:31.527743 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa46af1-b612-4da5-b7db-cdb169707c7a None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:31.539577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc698a-45af-4ee0-9893-593c4f2a06e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:31.540686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc698a-45af-4ee0-9893-593c4f2a06e2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:31.541169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc698a-45af-4ee0-9893-593c4f2a06e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:31.541737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc698a-45af-4ee0-9893-593c4f2a06e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:31.555167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc698a-45af-4ee0-9893-593c4f2a06e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:31.556686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc698a-45af-4ee0-9893-593c4f2a06e2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:31.567184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de50e8-1958-4b0e-9878-e12ffa09312d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:31.567836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de50e8-1958-4b0e-9878-e12ffa09312d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:31.568120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de50e8-1958-4b0e-9878-e12ffa09312d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:31.568397 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de50e8-1958-4b0e-9878-e12ffa09312d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:31.575244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-b0dc698a-45af-4ee0-9893-593c4f2a06e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:31.575244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:31.575244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:31.575244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:31.575244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:31.575244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:31.575244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:31.575244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:31.575244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:31.575244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:31.575244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:31.575244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:31.575244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:31.575244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:31.575244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:31.576111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:31.576981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:31.576981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:31.576981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:31.576981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:31.581068 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 323/662] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:31.582155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc698a-45af-4ee0-9893-593c4f2a06e2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:31.645297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de50e8-1958-4b0e-9878-e12ffa09312d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:31.646853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de50e8-1958-4b0e-9878-e12ffa09312d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:31.689446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc698a-45af-4ee0-9893-593c4f2a06e2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:31.697027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:31.727750 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17a7df-9a8e-4a78-883b-f6644e113193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:31.728548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17a7df-9a8e-4a78-883b-f6644e113193 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:31.728853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17a7df-9a8e-4a78-883b-f6644e113193 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:31.729104 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17a7df-9a8e-4a78-883b-f6644e113193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:31.741227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17a7df-9a8e-4a78-883b-f6644e113193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:31.742672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17a7df-9a8e-4a78-883b-f6644e113193 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:31.759908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ef17a7df-9a8e-4a78-883b-f6644e113193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:31.759908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:31.759908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:31.759908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:31.759908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:31.759908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:31.759908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:31.759908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:31.759908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:31.759908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:31.759908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:31.759908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:31.759908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:31.759908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:31.759908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:31.760822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:31.761781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:31.761781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:31.761781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:31.761781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:31.761781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 324/663] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:31.762366 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5531798-97cb-40c8-a3b6-1a05c6826058 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:31.762647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5531798-97cb-40c8-a3b6-1a05c6826058 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:31.762926 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5531798-97cb-40c8-a3b6-1a05c6826058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:31.770290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 340/664] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:17:31.771345 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de50e8-1958-4b0e-9878-e12ffa09312d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:31.780656 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e5531798-97cb-40c8-a3b6-1a05c6826058 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:31.781886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e5531798-97cb-40c8-a3b6-1a05c6826058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:31.790360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:31.857447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 325/665] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:31.858622 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5531798-97cb-40c8-a3b6-1a05c6826058 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:31.882199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de50e8-1958-4b0e-9878-e12ffa09312d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:31.915872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff076e1-1adc-4e94-a39a-2a83b1b6d13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:31.916469 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff076e1-1adc-4e94-a39a-2a83b1b6d13a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:31.916755 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff076e1-1adc-4e94-a39a-2a83b1b6d13a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:31.917031 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff076e1-1adc-4e94-a39a-2a83b1b6d13a None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:31.917803 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff076e1-1adc-4e94-a39a-2a83b1b6d13a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:31.918113 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff076e1-1adc-4e94-a39a-2a83b1b6d13a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:31.919527 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff076e1-1adc-4e94-a39a-2a83b1b6d13a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:31.935898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 341/666] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:31.936734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff076e1-1adc-4e94-a39a-2a83b1b6d13a None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:31.939669 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:31.975280 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5531798-97cb-40c8-a3b6-1a05c6826058 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:32.009732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23271c32-21ba-464d-8222-690e8b291ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:32.010384 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23271c32-21ba-464d-8222-690e8b291ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:32.010666 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23271c32-21ba-464d-8222-690e8b291ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:32.010935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23271c32-21ba-464d-8222-690e8b291ac0 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:32.011654 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23271c32-21ba-464d-8222-690e8b291ac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:32.011957 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23271c32-21ba-464d-8222-690e8b291ac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:32.013336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23271c32-21ba-464d-8222-690e8b291ac0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:32.024948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 326/667] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:32.026021 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23271c32-21ba-464d-8222-690e8b291ac0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:32.028665 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:32.051091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff076e1-1adc-4e94-a39a-2a83b1b6d13a None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:32.087955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48042e6-db88-4039-ba87-c14b28d3dac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:32.088739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48042e6-db88-4039-ba87-c14b28d3dac5 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:32.089022 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48042e6-db88-4039-ba87-c14b28d3dac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:32.089330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48042e6-db88-4039-ba87-c14b28d3dac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:32.100518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48042e6-db88-4039-ba87-c14b28d3dac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:32.102011 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48042e6-db88-4039-ba87-c14b28d3dac5 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:32.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-e48042e6-db88-4039-ba87-c14b28d3dac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:32.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:32.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:32.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:32.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:32.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:32.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:32.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:32.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:32.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:32.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:32.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:32.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:32.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:32.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:32.120311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:32.121236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:32.121236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:32.121236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:32.121236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:32.121236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 342/668] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:32.121236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48042e6-db88-4039-ba87-c14b28d3dac5 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:32.139009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23271c32-21ba-464d-8222-690e8b291ac0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:32.176281 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1d0af-07c6-4951-8b1a-bd64a29f00dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:32.177421 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1d0af-07c6-4951-8b1a-bd64a29f00dd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:32.177740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1d0af-07c6-4951-8b1a-bd64a29f00dd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:32.177990 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1d0af-07c6-4951-8b1a-bd64a29f00dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:32.186055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:32.188610 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1d0af-07c6-4951-8b1a-bd64a29f00dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-70b1d0af-07c6-4951-8b1a-bd64a29f00dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:32.197105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:32.198075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:32.198941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:32.198941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:32.198941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:32.198941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:32.198941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:32.198941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:32.198941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:32.198941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:32.198941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:32.198941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:32.198941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:32.198941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:32.198941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 327/669] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:17:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:32.199632 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1d0af-07c6-4951-8b1a-bd64a29f00dd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:32.241877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48042e6-db88-4039-ba87-c14b28d3dac5 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:32.278425 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfbf97-047e-4357-9f7a-f2e218332338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:32.279407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfbf97-047e-4357-9f7a-f2e218332338 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:32.279749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfbf97-047e-4357-9f7a-f2e218332338 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:32.280085 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfbf97-047e-4357-9f7a-f2e218332338 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:32.281521 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:32.289407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cfbf97-047e-4357-9f7a-f2e218332338 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:32.291064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cfbf97-047e-4357-9f7a-f2e218332338 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:32.308224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-e6cfbf97-047e-4357-9f7a-f2e218332338 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:32.308224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:32.308224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:32.308224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:32.308224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:32.308224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:32.308224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:32.308224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:32.308224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:32.308224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:32.308224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:32.308224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:32.308224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:32.308224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:32.308224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:32.309206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:32.310106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:32.310106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:32.310106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:32.310106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:32.310106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:32.310106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:32.310106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:32.310106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:32.310106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 343/670] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:32.310599 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfbf97-047e-4357-9f7a-f2e218332338 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:32.334225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1d0af-07c6-4951-8b1a-bd64a29f00dd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:32.386944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c83b01-c60c-4c9d-9b7b-8aea8728b1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:32.387811 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c83b01-c60c-4c9d-9b7b-8aea8728b1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:32.388145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c83b01-c60c-4c9d-9b7b-8aea8728b1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:32.388632 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c83b01-c60c-4c9d-9b7b-8aea8728b1b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:32.401001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c83b01-c60c-4c9d-9b7b-8aea8728b1b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:32.403495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c83b01-c60c-4c9d-9b7b-8aea8728b1b8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:32.418904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-a1c83b01-c60c-4c9d-9b7b-8aea8728b1b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:32.418904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:32.418904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:32.418904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:32.418904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:32.418904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:32.418904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:32.418904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:32.418904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:32.418904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:32.418904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:32.418904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:32.418904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:32.418904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:32.418904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:32.420005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:32.421216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:32.421216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:32.421216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:32.421216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:32.421216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:32.421216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:32.421216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:32.421216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:32.425505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 328/671] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:32.427123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c83b01-c60c-4c9d-9b7b-8aea8728b1b8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:32.464981 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfbf97-047e-4357-9f7a-f2e218332338 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:32.485033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:32.509162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acad92d-e7f9-4641-85ad-1e28f20fc0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:32.509854 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acad92d-e7f9-4641-85ad-1e28f20fc0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:32.510130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acad92d-e7f9-4641-85ad-1e28f20fc0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:32.510411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acad92d-e7f9-4641-85ad-1e28f20fc0f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:32.511587 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acad92d-e7f9-4641-85ad-1e28f20fc0f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:32.511587 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acad92d-e7f9-4641-85ad-1e28f20fc0f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:32.512916 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acad92d-e7f9-4641-85ad-1e28f20fc0f5 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:32.531054 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 344/672] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:32.531054 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acad92d-e7f9-4641-85ad-1e28f20fc0f5 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:32.552790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c83b01-c60c-4c9d-9b7b-8aea8728b1b8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:32.579789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:32.593402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010446d0-0f4e-468f-8971-bc249ccc1902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:32.594192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010446d0-0f4e-468f-8971-bc249ccc1902 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:32.594485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010446d0-0f4e-468f-8971-bc249ccc1902 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:32.594754 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010446d0-0f4e-468f-8971-bc249ccc1902 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:32.595508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010446d0-0f4e-468f-8971-bc249ccc1902 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:32.595815 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010446d0-0f4e-468f-8971-bc249ccc1902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:32.597218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010446d0-0f4e-468f-8971-bc249ccc1902 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:32.610119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 329/673] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:32.610119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010446d0-0f4e-468f-8971-bc249ccc1902 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:32.654354 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acad92d-e7f9-4641-85ad-1e28f20fc0f5 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:32.693070 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b6a45-0c9f-4901-8fe2-bb5f2bae2f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:32.694224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b6a45-0c9f-4901-8fe2-bb5f2bae2f84 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:32.694506 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b6a45-0c9f-4901-8fe2-bb5f2bae2f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:32.694779 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b6a45-0c9f-4901-8fe2-bb5f2bae2f84 None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:32.695581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b6a45-0c9f-4901-8fe2-bb5f2bae2f84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:32.695889 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b6a45-0c9f-4901-8fe2-bb5f2bae2f84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:32.699091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b6a45-0c9f-4901-8fe2-bb5f2bae2f84 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:32.714357 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 345/674] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:32.714975 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b6a45-0c9f-4901-8fe2-bb5f2bae2f84 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:32.726479 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010446d0-0f4e-468f-8971-bc249ccc1902 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:32.754997 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:32.762381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd23ff6-5b87-43aa-aba3-3f2ae542e346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:32.763070 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd23ff6-5b87-43aa-aba3-3f2ae542e346 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:32.763354 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd23ff6-5b87-43aa-aba3-3f2ae542e346 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:32.763629 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd23ff6-5b87-43aa-aba3-3f2ae542e346 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:32.764423 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd23ff6-5b87-43aa-aba3-3f2ae542e346 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:32.764733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd23ff6-5b87-43aa-aba3-3f2ae542e346 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:32.766189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd23ff6-5b87-43aa-aba3-3f2ae542e346 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:32.779371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 330/675] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:32.780201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd23ff6-5b87-43aa-aba3-3f2ae542e346 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:32.835033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:32.839149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b6a45-0c9f-4901-8fe2-bb5f2bae2f84 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:32.878910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a788b6e-1c5c-4c72-88dd-1ccd5b49792b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:32.879713 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a788b6e-1c5c-4c72-88dd-1ccd5b49792b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:32.880000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a788b6e-1c5c-4c72-88dd-1ccd5b49792b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:32.880281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a788b6e-1c5c-4c72-88dd-1ccd5b49792b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:32.887379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a788b6e-1c5c-4c72-88dd-1ccd5b49792b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:32.888838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a788b6e-1c5c-4c72-88dd-1ccd5b49792b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:32.893815 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd23ff6-5b87-43aa-aba3-3f2ae542e346 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:32.903044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-9a788b6e-1c5c-4c72-88dd-1ccd5b49792b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:32.903044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:32.903044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:32.903044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:32.903044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:32.903044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:32.903044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:32.903044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:32.903044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:32.903044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:32.903044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:32.903044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:32.903044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:32.903044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:32.903044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:32.904131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:32.911481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:32.911481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:32.911481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:32.911481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:32.911481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:32.911481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:32.911481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:32.911481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:32.911481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 346/676] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:32.911481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545de25d-0fd4-42f0-a8ec-38425105236f None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:32.911481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545de25d-0fd4-42f0-a8ec-38425105236f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:32.912494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545de25d-0fd4-42f0-a8ec-38425105236f None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:32.912494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 347/677] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:32.912494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545de25d-0fd4-42f0-a8ec-38425105236f None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:32.941883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6d960-0c21-4bb8-b869-9c85ad27aaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:32.942682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6d960-0c21-4bb8-b869-9c85ad27aaf5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:32.942971 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6d960-0c21-4bb8-b869-9c85ad27aaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:32.943298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6d960-0c21-4bb8-b869-9c85ad27aaf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:32.954185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6d960-0c21-4bb8-b869-9c85ad27aaf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:32.955661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6d960-0c21-4bb8-b869-9c85ad27aaf5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:32.975445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-02f6d960-0c21-4bb8-b869-9c85ad27aaf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:32.975445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:32.975445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:32.975445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:32.975445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:32.975445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:32.975445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:32.975445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:32.975445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:32.975445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:32.975445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:32.975445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:32.975445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:32.975445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:32.975445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:32.976709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:32.982390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:32.982390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:32.982390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:32.982390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:32.982390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 331/678] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:32.982390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6d960-0c21-4bb8-b869-9c85ad27aaf5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:33.011998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:33.027283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545de25d-0fd4-42f0-a8ec-38425105236f None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:33.070908 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a4837-883e-4810-957c-3fbbeb8e18b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:33.071950 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a4837-883e-4810-957c-3fbbeb8e18b5 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:33.072422 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a4837-883e-4810-957c-3fbbeb8e18b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:33.072889 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a4837-883e-4810-957c-3fbbeb8e18b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:33.074455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a4837-883e-4810-957c-3fbbeb8e18b5 None admin] RBAC: Adding query filter params (role.id=c8578ede4e7041ecaeba79c255cc650b, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=215c6f6f1aad486f85ce20590d889cbc) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:33.074983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a4837-883e-4810-957c-3fbbeb8e18b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:33.077461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a4837-883e-4810-957c-3fbbeb8e18b5 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:33.079185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:33.084632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 348/679] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:32 2022] GET /identity/v3/role_assignments?user.id=215c6f6f1aad486f85ce20590d889cbc&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=c8578ede4e7041ecaeba79c255cc650b => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:33.085681 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a4837-883e-4810-957c-3fbbeb8e18b5 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:33.086352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6d960-0c21-4bb8-b869-9c85ad27aaf5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:33.145244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff4c8f-e62b-4e60-9c00-0e7467f48929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:33.145936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff4c8f-e62b-4e60-9c00-0e7467f48929 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:33.146348 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff4c8f-e62b-4e60-9c00-0e7467f48929 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:33.146627 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff4c8f-e62b-4e60-9c00-0e7467f48929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:33.148642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff4c8f-e62b-4e60-9c00-0e7467f48929 None admin] RBAC: Adding query filter params (role.id=c8578ede4e7041ecaeba79c255cc650b, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=7245ef27d2454927a01a7cd56a4c27aa) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:33.148971 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff4c8f-e62b-4e60-9c00-0e7467f48929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:33.150940 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff4c8f-e62b-4e60-9c00-0e7467f48929 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:33.169540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 332/680] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:32 2022] GET /identity/v3/role_assignments?user.id=7245ef27d2454927a01a7cd56a4c27aa&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=c8578ede4e7041ecaeba79c255cc650b => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:33.169540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f16c4f-4403-471f-bcb6-920273d8bcf7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:33.169540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f16c4f-4403-471f-bcb6-920273d8bcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:33.169540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f16c4f-4403-471f-bcb6-920273d8bcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:33.189956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-17f16c4f-4403-471f-bcb6-920273d8bcf7 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:33.191661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-17f16c4f-4403-471f-bcb6-920273d8bcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:33.244455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a4837-883e-4810-957c-3fbbeb8e18b5 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:33.304131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 333/681] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:33.304131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f16c4f-4403-471f-bcb6-920273d8bcf7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:33.313952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160099da-5f2a-4c26-9ce7-508d9d2ab4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:33.315002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160099da-5f2a-4c26-9ce7-508d9d2ab4eb None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:33.316096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160099da-5f2a-4c26-9ce7-508d9d2ab4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:33.316593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160099da-5f2a-4c26-9ce7-508d9d2ab4eb None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:33.318507 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160099da-5f2a-4c26-9ce7-508d9d2ab4eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:33.318507 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160099da-5f2a-4c26-9ce7-508d9d2ab4eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:33.319423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160099da-5f2a-4c26-9ce7-508d9d2ab4eb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:33.339490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 349/682] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:33.340806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160099da-5f2a-4c26-9ce7-508d9d2ab4eb None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:33.371086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:33.425377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:33.435530 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f16c4f-4403-471f-bcb6-920273d8bcf7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:33.477566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160099da-5f2a-4c26-9ce7-508d9d2ab4eb None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:33.484139 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d8685-f865-42d1-92dd-de2ae72bf8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:33.485049 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d8685-f865-42d1-92dd-de2ae72bf8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:33.485549 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d8685-f865-42d1-92dd-de2ae72bf8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:33.486545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d8685-f865-42d1-92dd-de2ae72bf8a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:33.488156 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d8685-f865-42d1-92dd-de2ae72bf8a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:33.489132 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d8685-f865-42d1-92dd-de2ae72bf8a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:33.491186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d8685-f865-42d1-92dd-de2ae72bf8a9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:33.513412 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 334/683] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:33.513412 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d8685-f865-42d1-92dd-de2ae72bf8a9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:33.532896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa5591-bab0-4019-a1ed-054f745343f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:33.534849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa5591-bab0-4019-a1ed-054f745343f1 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:33.535543 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa5591-bab0-4019-a1ed-054f745343f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:33.535972 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa5591-bab0-4019-a1ed-054f745343f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:33.551985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa5591-bab0-4019-a1ed-054f745343f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-95aa5591-bab0-4019-a1ed-054f745343f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:33.563028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:33.564220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:33.565327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:33.565327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:33.565327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:33.565327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:33.565327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:33.565327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:33.565327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:33.565327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:33.565327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:33.565327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:33.565327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:33.565327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:33.568005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 350/684] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:33.569773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa5591-bab0-4019-a1ed-054f745343f1 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:33.636533 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d8685-f865-42d1-92dd-de2ae72bf8a9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:33.667395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:33.688946 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1087dd-6d50-410d-94b7-e6bdabd42390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:33.690216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1087dd-6d50-410d-94b7-e6bdabd42390 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:33.690658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1087dd-6d50-410d-94b7-e6bdabd42390 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:33.691529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1087dd-6d50-410d-94b7-e6bdabd42390 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:33.692827 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1087dd-6d50-410d-94b7-e6bdabd42390 None admin] RBAC: Adding query filter params (role.id=c8578ede4e7041ecaeba79c255cc650b, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=369bdaded0534142b03ad3cf43a70881) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:33.693426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1087dd-6d50-410d-94b7-e6bdabd42390 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:33.695213 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1087dd-6d50-410d-94b7-e6bdabd42390 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:33.695488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa5591-bab0-4019-a1ed-054f745343f1 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:33.703733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 335/685] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:33 2022] GET /identity/v3/role_assignments?user.id=369bdaded0534142b03ad3cf43a70881&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=c8578ede4e7041ecaeba79c255cc650b => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:33.705433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1087dd-6d50-410d-94b7-e6bdabd42390 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:33.729090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:33.736679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf96c9-442e-47bc-bfb4-f3bf496eb6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:33.738380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf96c9-442e-47bc-bfb4-f3bf496eb6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:33.738837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf96c9-442e-47bc-bfb4-f3bf496eb6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:33.739220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf96c9-442e-47bc-bfb4-f3bf496eb6a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:33.748828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cf96c9-442e-47bc-bfb4-f3bf496eb6a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:33.750622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cf96c9-442e-47bc-bfb4-f3bf496eb6a6 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-c8cf96c9-442e-47bc-bfb4-f3bf496eb6a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:33.768558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:33.769730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:33.770696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:33.770696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:33.770696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:33.770696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:33.770696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:33.770696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:33.770696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:33.770696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:33.770696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:17:33.770696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:33.771235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 351/686] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Jul 19 15:17:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:33.776692 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf96c9-442e-47bc-bfb4-f3bf496eb6a6 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:33.846227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1087dd-6d50-410d-94b7-e6bdabd42390 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:33.890856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12e3a7-867d-4c3b-afe2-59069ed539c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:33.892093 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12e3a7-867d-4c3b-afe2-59069ed539c5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:33.892583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12e3a7-867d-4c3b-afe2-59069ed539c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:33.893045 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12e3a7-867d-4c3b-afe2-59069ed539c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:33.894188 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12e3a7-867d-4c3b-afe2-59069ed539c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:33.894700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12e3a7-867d-4c3b-afe2-59069ed539c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:33.896297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12e3a7-867d-4c3b-afe2-59069ed539c5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:33.907781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 336/687] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:33.918202 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12e3a7-867d-4c3b-afe2-59069ed539c5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:33.965167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf96c9-442e-47bc-bfb4-f3bf496eb6a6 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:34.004340 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7758e6-0995-4732-b2c5-331d190035f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:34.005300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7758e6-0995-4732-b2c5-331d190035f8 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:34.005300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7758e6-0995-4732-b2c5-331d190035f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:34.005588 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7758e6-0995-4732-b2c5-331d190035f8 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:34.006447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7758e6-0995-4732-b2c5-331d190035f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:34.006756 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7758e6-0995-4732-b2c5-331d190035f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:34.008202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7758e6-0995-4732-b2c5-331d190035f8 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:34.017573 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:34.030786 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 352/688] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Jul 19 15:17:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:34.035362 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:34.037265 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7758e6-0995-4732-b2c5-331d190035f8 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:34.061517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12e3a7-867d-4c3b-afe2-59069ed539c5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:34.098057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22abd27-df81-49e2-8c80-7a4869eeb6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:34.099209 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22abd27-df81-49e2-8c80-7a4869eeb6db None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:34.099713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22abd27-df81-49e2-8c80-7a4869eeb6db None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:34.100178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22abd27-df81-49e2-8c80-7a4869eeb6db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:34.111745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22abd27-df81-49e2-8c80-7a4869eeb6db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:34.113975 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22abd27-df81-49e2-8c80-7a4869eeb6db None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:34.132085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f22abd27-df81-49e2-8c80-7a4869eeb6db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:34.132085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:34.132085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:34.132085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:34.132085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:34.132085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:34.132085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:34.132085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:34.132085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:34.132085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:34.132085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:34.132085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:34.132085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:34.132085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:34.132085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:34.132987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:34.133892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:34.133892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:34.133892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:34.133892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:34.136124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 337/689] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:34.141284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22abd27-df81-49e2-8c80-7a4869eeb6db None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:34.147672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7758e6-0995-4732-b2c5-331d190035f8 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:34.187480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41ccd9-2aea-4462-81f4-715d4ba9891e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:34.188136 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41ccd9-2aea-4462-81f4-715d4ba9891e None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:34.188416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41ccd9-2aea-4462-81f4-715d4ba9891e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:34.188690 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41ccd9-2aea-4462-81f4-715d4ba9891e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:34.250123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22abd27-df81-49e2-8c80-7a4869eeb6db None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:34.255142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:34.263013 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41ccd9-2aea-4462-81f4-715d4ba9891e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:34.264492 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41ccd9-2aea-4462-81f4-715d4ba9891e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:34.280560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:34.289920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689b4de-1016-4565-b9f5-eedae4fc646c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:34.290904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689b4de-1016-4565-b9f5-eedae4fc646c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:34.291434 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689b4de-1016-4565-b9f5-eedae4fc646c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:34.291906 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689b4de-1016-4565-b9f5-eedae4fc646c None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:34.292828 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689b4de-1016-4565-b9f5-eedae4fc646c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:34.293367 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689b4de-1016-4565-b9f5-eedae4fc646c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:34.294937 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689b4de-1016-4565-b9f5-eedae4fc646c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:34.308113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 338/690] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:34.314262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689b4de-1016-4565-b9f5-eedae4fc646c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:34.378650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 353/691] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:17:34.388038 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41ccd9-2aea-4462-81f4-715d4ba9891e None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:34.445094 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689b4de-1016-4565-b9f5-eedae4fc646c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:34.482534 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbdd3ef-4953-4a35-bd88-87dc6fcf033c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:34.483706 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbdd3ef-4953-4a35-bd88-87dc6fcf033c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:34.484189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbdd3ef-4953-4a35-bd88-87dc6fcf033c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:34.484658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbdd3ef-4953-4a35-bd88-87dc6fcf033c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:34.493089 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbdd3ef-4953-4a35-bd88-87dc6fcf033c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:34.494818 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbdd3ef-4953-4a35-bd88-87dc6fcf033c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3fbdd3ef-4953-4a35-bd88-87dc6fcf033c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:34.510379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:34.511433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:34.512352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:34.512352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:34.512352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:34.512352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:34.512352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:34.512352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:34.512352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:34.512352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:34.512352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 15:17:34.512352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:34.514932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 339/692] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:17:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:34.520595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbdd3ef-4953-4a35-bd88-87dc6fcf033c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:34.583704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:34.590626 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41ccd9-2aea-4462-81f4-715d4ba9891e None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:34.630244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bba29-4c5b-4e6d-b8b5-70ac8b139e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:34.631826 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bba29-4c5b-4e6d-b8b5-70ac8b139e9f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:34.632311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bba29-4c5b-4e6d-b8b5-70ac8b139e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:34.632816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bba29-4c5b-4e6d-b8b5-70ac8b139e9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:34.639374 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:34.647537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bba29-4c5b-4e6d-b8b5-70ac8b139e9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:34.649425 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bba29-4c5b-4e6d-b8b5-70ac8b139e9f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:34.675647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-a25bba29-4c5b-4e6d-b8b5-70ac8b139e9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:34.675647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:34.675647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:34.675647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:34.675647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:34.675647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:34.675647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:34.675647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:34.675647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:34.675647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:34.675647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:34.675647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:34.675647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:34.675647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:34.675647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:34.676582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:34.677487 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:34.677487 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:34.677487 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:34.677487 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:34.682663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 354/693] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:34.687858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bba29-4c5b-4e6d-b8b5-70ac8b139e9f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:34.726272 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbdd3ef-4953-4a35-bd88-87dc6fcf033c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:34.802322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9049a5-8042-4899-82ef-62c6a95db603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:34.803379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9049a5-8042-4899-82ef-62c6a95db603 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:34.804491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9049a5-8042-4899-82ef-62c6a95db603 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:34.804967 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9049a5-8042-4899-82ef-62c6a95db603 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:34.806174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9049a5-8042-4899-82ef-62c6a95db603 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:34.807766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9049a5-8042-4899-82ef-62c6a95db603 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:34.810095 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9049a5-8042-4899-82ef-62c6a95db603 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:34.842875 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 340/694] 10.222.0.42 () {56 vars in 1135 bytes} [Tue Jul 19 15:17:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:34.848856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9049a5-8042-4899-82ef-62c6a95db603 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:34.877725 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bba29-4c5b-4e6d-b8b5-70ac8b139e9f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:34.943969 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af552d-193b-4490-bec5-01e18f3196a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:34.945020 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af552d-193b-4490-bec5-01e18f3196a3 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:34.945533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af552d-193b-4490-bec5-01e18f3196a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:34.946179 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af552d-193b-4490-bec5-01e18f3196a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:34.947773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af552d-193b-4490-bec5-01e18f3196a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:34.948716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af552d-193b-4490-bec5-01e18f3196a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:34.950490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af552d-193b-4490-bec5-01e18f3196a3 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:34.976472 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 355/695] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:34.983075 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af552d-193b-4490-bec5-01e18f3196a3 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:34.998386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:35.037896 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9049a5-8042-4899-82ef-62c6a95db603 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:35.055906 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:35.100711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb2eb6-83e3-4405-a0de-0bf5c3f67174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:35.101318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb2eb6-83e3-4405-a0de-0bf5c3f67174 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:35.101608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb2eb6-83e3-4405-a0de-0bf5c3f67174 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:35.101889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb2eb6-83e3-4405-a0de-0bf5c3f67174 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:35.156498 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af552d-193b-4490-bec5-01e18f3196a3 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:35.216864 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7406ff46-af8b-429e-b4e3-8efb32f2f5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:35.218053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406ff46-af8b-429e-b4e3-8efb32f2f5aa None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:35.218538 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406ff46-af8b-429e-b4e3-8efb32f2f5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:35.219010 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406ff46-af8b-429e-b4e3-8efb32f2f5aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:35.226010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb2eb6-83e3-4405-a0de-0bf5c3f67174 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:35.231000 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb2eb6-83e3-4405-a0de-0bf5c3f67174 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:35.234162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406ff46-af8b-429e-b4e3-8efb32f2f5aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:35.236589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406ff46-af8b-429e-b4e3-8efb32f2f5aa None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:35.260691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-7406ff46-af8b-429e-b4e3-8efb32f2f5aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:35.260691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:35.260691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:35.260691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:35.260691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:35.260691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:35.260691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:35.260691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:35.260691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:35.260691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:35.260691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:35.260691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:35.260691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:35.260691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:35.260691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:35.261607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:35.262506 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:35.262506 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:35.262506 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:35.262506 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:35.262506 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:35.262506 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:35.262506 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:35.262506 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:35.263682 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 356/696] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:35.271398 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7406ff46-af8b-429e-b4e3-8efb32f2f5aa None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:35.385103 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 341/697] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 15:17:35.386427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb2eb6-83e3-4405-a0de-0bf5c3f67174 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:35.397863 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:35.421477 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:35.464212 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7406ff46-af8b-429e-b4e3-8efb32f2f5aa None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:35.503612 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb2eb6-83e3-4405-a0de-0bf5c3f67174 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:35.508843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6dda3-eee6-477a-9a42-2278b624e73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:35.509549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6dda3-eee6-477a-9a42-2278b624e73d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:35.509832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6dda3-eee6-477a-9a42-2278b624e73d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:35.510106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6dda3-eee6-477a-9a42-2278b624e73d None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:35.510902 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6dda3-eee6-477a-9a42-2278b624e73d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:35.511212 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6dda3-eee6-477a-9a42-2278b624e73d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:35.512641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6dda3-eee6-477a-9a42-2278b624e73d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:35.535100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 357/698] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:35.535100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a86802-b4c1-43cc-9fea-7a4a10dc06c9 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:35.535100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a86802-b4c1-43cc-9fea-7a4a10dc06c9 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:35.535100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a86802-b4c1-43cc-9fea-7a4a10dc06c9 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:35.535100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 358/699] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:35.536057 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a86802-b4c1-43cc-9fea-7a4a10dc06c9 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:35.548891 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca4d82-ee4a-44b1-b89c-c8abdcbd4aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:35.550748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca4d82-ee4a-44b1-b89c-c8abdcbd4aba None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:35.551243 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca4d82-ee4a-44b1-b89c-c8abdcbd4aba None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:35.551720 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca4d82-ee4a-44b1-b89c-c8abdcbd4aba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:35.562887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca4d82-ee4a-44b1-b89c-c8abdcbd4aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:35.564644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca4d82-ee4a-44b1-b89c-c8abdcbd4aba None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:35.585170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-6fca4d82-ee4a-44b1-b89c-c8abdcbd4aba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:35.585170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:35.585170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:35.585170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:35.585170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:35.585170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:35.585170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:35.585170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:35.585170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:35.585170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:35.585170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:35.585170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:35.585170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:35.585170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:35.585170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:35.586340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:35.587494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:35.587494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:35.587494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:35.587494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:35.589590 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 342/700] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:35.591083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898d08d-23ec-4911-a803-e5d4fd8f8f7b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:35.591589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898d08d-23ec-4911-a803-e5d4fd8f8f7b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:35.592082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898d08d-23ec-4911-a803-e5d4fd8f8f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:35.615006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c898d08d-23ec-4911-a803-e5d4fd8f8f7b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:35.616827 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c898d08d-23ec-4911-a803-e5d4fd8f8f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:35.658246 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a86802-b4c1-43cc-9fea-7a4a10dc06c9 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:35.684760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:35.688594 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:35.698429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a3c6b-6c00-465a-b7c3-2de4a708fdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:35.699084 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a3c6b-6c00-465a-b7c3-2de4a708fdec None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:35.699364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a3c6b-6c00-465a-b7c3-2de4a708fdec None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:35.699636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a3c6b-6c00-465a-b7c3-2de4a708fdec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:35.705244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a3c6b-6c00-465a-b7c3-2de4a708fdec None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=bff6a2d4bc734904b2615608a86b097d) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:35.705383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 343/701] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:35.705472 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a3c6b-6c00-465a-b7c3-2de4a708fdec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:35.706279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898d08d-23ec-4911-a803-e5d4fd8f8f7b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:35.706521 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a3c6b-6c00-465a-b7c3-2de4a708fdec None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:35.717751 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 359/702] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:35 2022] GET /identity/v3/role_assignments?user.id=bff6a2d4bc734904b2615608a86b097d&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:35.719262 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528f246-8e5c-42d6-b214-669117f5c67f None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:35.719548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528f246-8e5c-42d6-b214-669117f5c67f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:35.719824 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528f246-8e5c-42d6-b214-669117f5c67f None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:35.724770 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 360/703] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:35.730736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c89e38-07e8-448a-8656-2a610cb4b746 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:35.731130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c89e38-07e8-448a-8656-2a610cb4b746 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:35.731478 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c89e38-07e8-448a-8656-2a610cb4b746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:35.754578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-d0c89e38-07e8-448a-8656-2a610cb4b746 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:35.756315 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d0c89e38-07e8-448a-8656-2a610cb4b746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:35.837154 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898d08d-23ec-4911-a803-e5d4fd8f8f7b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:35.845270 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 361/704] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:35.846821 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c89e38-07e8-448a-8656-2a610cb4b746 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:35.884551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd904b-9a31-498a-8ae3-78c685fcf4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:35.885548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd904b-9a31-498a-8ae3-78c685fcf4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:35.885973 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd904b-9a31-498a-8ae3-78c685fcf4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:35.886499 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd904b-9a31-498a-8ae3-78c685fcf4f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:35.887490 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd904b-9a31-498a-8ae3-78c685fcf4f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:35.887951 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd904b-9a31-498a-8ae3-78c685fcf4f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:35.889369 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd904b-9a31-498a-8ae3-78c685fcf4f4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:35.903626 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 344/705] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:35.904937 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd904b-9a31-498a-8ae3-78c685fcf4f4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:35.959244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:35.983257 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c89e38-07e8-448a-8656-2a610cb4b746 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:36.015169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:36.021493 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd904b-9a31-498a-8ae3-78c685fcf4f4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:36.032414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a9af2-3320-4cf0-b94b-ca20060be1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:36.034081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a9af2-3320-4cf0-b94b-ca20060be1fc None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:36.034518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a9af2-3320-4cf0-b94b-ca20060be1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:36.035078 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a9af2-3320-4cf0-b94b-ca20060be1fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:36.049412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a9af2-3320-4cf0-b94b-ca20060be1fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:36.051439 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a9af2-3320-4cf0-b94b-ca20060be1fc None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:36.060696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c26c3-08b3-44e3-a6b3-82d7a1322e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:36.061746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c26c3-08b3-44e3-a6b3-82d7a1322e3a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:36.062129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c26c3-08b3-44e3-a6b3-82d7a1322e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:36.062653 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c26c3-08b3-44e3-a6b3-82d7a1322e3a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:36.063453 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c26c3-08b3-44e3-a6b3-82d7a1322e3a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:36.066230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c26c3-08b3-44e3-a6b3-82d7a1322e3a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:36.075529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-3d6a9af2-3320-4cf0-b94b-ca20060be1fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:36.075529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:36.075529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:36.075529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:36.075529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:36.075529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:36.075529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:36.075529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:36.075529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:36.075529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:36.075529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:36.075529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:36.075529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:36.075529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:36.075529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:36.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:36.077778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:36.077778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:36.077778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:36.077778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:36.079672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 362/706] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:36.081541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a9af2-3320-4cf0-b94b-ca20060be1fc None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-726c26c3-08b3-44e3-a6b3-82d7a1322e3a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:36.089934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:36.091145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:36.092221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:36.092221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:36.092221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:36.092221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:36.092221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:36.092221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:36.092221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:36.092221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:36.092221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:36.092221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:36.092221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:36.092221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:17:36.092221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:36.092221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 15:17:36.092221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:36.094908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 345/707] 10.222.0.42 () {56 vars in 1129 bytes} [Tue Jul 19 15:17:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:36.100119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c26c3-08b3-44e3-a6b3-82d7a1322e3a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:36.219573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c26c3-08b3-44e3-a6b3-82d7a1322e3a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:36.227869 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a9af2-3320-4cf0-b94b-ca20060be1fc None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:36.234308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:36.273786 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ba218-7885-480b-91ad-6655597aec28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:36.274593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ba218-7885-480b-91ad-6655597aec28 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:36.274879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ba218-7885-480b-91ad-6655597aec28 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:36.275160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ba218-7885-480b-91ad-6655597aec28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:36.276600 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbca81f-1df7-4987-8ffc-935fc6bf7181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:36.277672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbca81f-1df7-4987-8ffc-935fc6bf7181 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:36.278135 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbca81f-1df7-4987-8ffc-935fc6bf7181 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:36.278864 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbca81f-1df7-4987-8ffc-935fc6bf7181 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:36.279776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbca81f-1df7-4987-8ffc-935fc6bf7181 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:36.280261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbca81f-1df7-4987-8ffc-935fc6bf7181 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:36.281862 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbca81f-1df7-4987-8ffc-935fc6bf7181 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:36.283095 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ba218-7885-480b-91ad-6655597aec28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:36.284708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ba218-7885-480b-91ad-6655597aec28 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:36.296702 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 346/708] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:36.297751 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbca81f-1df7-4987-8ffc-935fc6bf7181 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:36.300922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-0c3ba218-7885-480b-91ad-6655597aec28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:36.300922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:36.300922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:36.300922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:36.300922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:36.300922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:36.300922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:36.300922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:36.300922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:36.300922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:36.300922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:36.300922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:36.300922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:36.300922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:36.300922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:36.302015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:36.302939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:36.302939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:36.302939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:36.302939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:36.302939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:36.302939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:36.302939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:36.302939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:36.307848 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 363/709] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:36.308784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ba218-7885-480b-91ad-6655597aec28 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:36.344896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:36.427652 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ba218-7885-480b-91ad-6655597aec28 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:36.430851 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbca81f-1df7-4987-8ffc-935fc6bf7181 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:36.465026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b21c03-5818-4ac1-ab7c-5f59d37f266b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:36.465718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b21c03-5818-4ac1-ab7c-5f59d37f266b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:36.465997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b21c03-5818-4ac1-ab7c-5f59d37f266b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:36.466267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b21c03-5818-4ac1-ab7c-5f59d37f266b None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:36.466969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b21c03-5818-4ac1-ab7c-5f59d37f266b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:36.467290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b21c03-5818-4ac1-ab7c-5f59d37f266b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:36.467386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485e109-7d0e-476e-89c0-35e32067ce3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:36.467999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485e109-7d0e-476e-89c0-35e32067ce3d None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:36.468287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485e109-7d0e-476e-89c0-35e32067ce3d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:36.468564 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485e109-7d0e-476e-89c0-35e32067ce3d None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:36.468668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b21c03-5818-4ac1-ab7c-5f59d37f266b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:36.469305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485e109-7d0e-476e-89c0-35e32067ce3d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:36.470988 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485e109-7d0e-476e-89c0-35e32067ce3d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:36.478093 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 347/710] 10.222.0.42 () {56 vars in 1124 bytes} [Tue Jul 19 15:17:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:36.478093 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b21c03-5818-4ac1-ab7c-5f59d37f266b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:36.493911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 364/711] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:17:36 2022] POST /identity/v3/users => generated 254 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:17:36.495487 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54421b2-bea6-43ff-891c-e581c65d8168 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:36.495778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54421b2-bea6-43ff-891c-e581c65d8168 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:36.496055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54421b2-bea6-43ff-891c-e581c65d8168 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:36.497652 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 365/712] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:36.498290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54421b2-bea6-43ff-891c-e581c65d8168 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:36.532252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:36.617546 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54421b2-bea6-43ff-891c-e581c65d8168 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:36.629684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:36.630569 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b21c03-5818-4ac1-ab7c-5f59d37f266b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:36.662135 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c368d7-07e5-40f4-980a-4a5d975ba3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:36.662829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c368d7-07e5-40f4-980a-4a5d975ba3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:36.663086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c368d7-07e5-40f4-980a-4a5d975ba3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:36.663363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c368d7-07e5-40f4-980a-4a5d975ba3b9 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:36.664273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c368d7-07e5-40f4-980a-4a5d975ba3b9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:36.664541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c368d7-07e5-40f4-980a-4a5d975ba3b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:36.666105 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c368d7-07e5-40f4-980a-4a5d975ba3b9 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:36.672131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 366/713] 10.222.0.42 () {56 vars in 1124 bytes} [Tue Jul 19 15:17:36 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:36.673089 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663cfa4-13cc-4505-a411-e058cdf8f6da None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:36.673415 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663cfa4-13cc-4505-a411-e058cdf8f6da None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:36.673694 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663cfa4-13cc-4505-a411-e058cdf8f6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:36.682235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dbad2-0e59-438a-baa9-9801bf479ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:36.683332 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dbad2-0e59-438a-baa9-9801bf479ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:36.683824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dbad2-0e59-438a-baa9-9801bf479ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:36.684319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dbad2-0e59-438a-baa9-9801bf479ff2 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:36.686032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482dbad2-0e59-438a-baa9-9801bf479ff2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:36.687282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482dbad2-0e59-438a-baa9-9801bf479ff2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:36.689175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482dbad2-0e59-438a-baa9-9801bf479ff2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:36.692820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-4663cfa4-13cc-4505-a411-e058cdf8f6da None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:36.694095 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-4663cfa4-13cc-4505-a411-e058cdf8f6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:36.707690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 348/714] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:36.714350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dbad2-0e59-438a-baa9-9801bf479ff2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:36.779055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 367/715] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:36.803733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663cfa4-13cc-4505-a411-e058cdf8f6da None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:36.842337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:36.854194 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dbad2-0e59-438a-baa9-9801bf479ff2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:36.896556 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80ff0e-a401-466e-a3e5-08074b062064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:36.897254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80ff0e-a401-466e-a3e5-08074b062064 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:36.897542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80ff0e-a401-466e-a3e5-08074b062064 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:36.897823 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80ff0e-a401-466e-a3e5-08074b062064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:36.898667 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80ff0e-a401-466e-a3e5-08074b062064 None admin] RBAC: Adding query filter params (role.id=c8578ede4e7041ecaeba79c255cc650b, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=96faaac55bc5408685c68a6177d91971) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:36.898989 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80ff0e-a401-466e-a3e5-08074b062064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:36.900418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80ff0e-a401-466e-a3e5-08074b062064 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:36.907303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 349/716] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:36 2022] GET /identity/v3/role_assignments?user.id=96faaac55bc5408685c68a6177d91971&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=c8578ede4e7041ecaeba79c255cc650b => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:36.917553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:36.938590 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663cfa4-13cc-4505-a411-e058cdf8f6da None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:36.977297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07cb37-cb2c-4085-bc54-49ee1a84964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:36.978411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07cb37-cb2c-4085-bc54-49ee1a84964c None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:36.978839 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07cb37-cb2c-4085-bc54-49ee1a84964c None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:36.979258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07cb37-cb2c-4085-bc54-49ee1a84964c None admin] PATH_INFO: `/v3/services/network` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:36.980160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07cb37-cb2c-4085-bc54-49ee1a84964c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:36.981834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07cb37-cb2c-4085-bc54-49ee1a84964c None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-be07cb37-cb2c-4085-bc54-49ee1a84964c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:36.997593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:36.998833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:36.999855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:36.999855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:36.999855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:36.999855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:36.999855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:36.999855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:36.999855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:36.999855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:36.999855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:36.999855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:36.999855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:17:36.999855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:36.999855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 15:17:36.999855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:37.003058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 368/717] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:36 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:37.009524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80ff0e-a401-466e-a3e5-08074b062064 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:37.163627 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80ff0e-a401-466e-a3e5-08074b062064 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:37.203102 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63699094-c7a2-4ea5-9cc0-e9fac49f9a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:37.203773 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63699094-c7a2-4ea5-9cc0-e9fac49f9a2b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:37.204059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63699094-c7a2-4ea5-9cc0-e9fac49f9a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:37.204339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63699094-c7a2-4ea5-9cc0-e9fac49f9a2b None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:37.205052 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63699094-c7a2-4ea5-9cc0-e9fac49f9a2b None admin] RBAC: Adding query filter params (name=network) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:37.205389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63699094-c7a2-4ea5-9cc0-e9fac49f9a2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:37.207423 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63699094-c7a2-4ea5-9cc0-e9fac49f9a2b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:37.216039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 350/718] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Jul 19 15:17:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:37.221968 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07cb37-cb2c-4085-bc54-49ee1a84964c None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:37.354189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07cb37-cb2c-4085-bc54-49ee1a84964c None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:37.391355 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b0715-a935-416f-99af-d45e23c68821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:37.392089 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b0715-a935-416f-99af-d45e23c68821 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:37.392464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b0715-a935-416f-99af-d45e23c68821 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:37.392769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b0715-a935-416f-99af-d45e23c68821 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:37.393612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b0715-a935-416f-99af-d45e23c68821 None admin] RBAC: Adding query filter params (type=network) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:37.393955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b0715-a935-416f-99af-d45e23c68821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:37.395511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b0715-a935-416f-99af-d45e23c68821 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:37.401223 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 369/719] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Jul 19 15:17:37 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:38.138068 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1808cf-3435-4f08-ae39-f9ff5d0ef38e None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:38.138596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1808cf-3435-4f08-ae39-f9ff5d0ef38e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:38.138596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1808cf-3435-4f08-ae39-f9ff5d0ef38e None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:38.140276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 351/720] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:38.145383 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e1fa1-6541-47d3-bad2-2c96579d0b90 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:38.145800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e1fa1-6541-47d3-bad2-2c96579d0b90 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:38.145977 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e1fa1-6541-47d3-bad2-2c96579d0b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:38.169166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-f19e1fa1-6541-47d3-bad2-2c96579d0b90 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:38.170379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f19e1fa1-6541-47d3-bad2-2c96579d0b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:38.184091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:38.261806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 370/721] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 15:17:38.296227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1808cf-3435-4f08-ae39-f9ff5d0ef38e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:38.328607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:38.426889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1808cf-3435-4f08-ae39-f9ff5d0ef38e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:38.473874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a326c39-0ffb-4d45-b7b0-5b9ff114424f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:38.474681 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a326c39-0ffb-4d45-b7b0-5b9ff114424f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:38.474966 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a326c39-0ffb-4d45-b7b0-5b9ff114424f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:38.475244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a326c39-0ffb-4d45-b7b0-5b9ff114424f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:38.492783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a326c39-0ffb-4d45-b7b0-5b9ff114424f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-5a326c39-0ffb-4d45-b7b0-5b9ff114424f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:38.500364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:38.501284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:38.502141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:38.502141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:38.502141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:38.502141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:38.502141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:38.502141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:38.502141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:38.502141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:38.502141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:38.502141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:38.502141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:38.502141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:38.503341 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 352/722] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:17:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:38.508118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e1fa1-6541-47d3-bad2-2c96579d0b90 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:38.617781 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e1fa1-6541-47d3-bad2-2c96579d0b90 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:38.652364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95f20b-eb79-4130-99a3-1712602aad21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:38.653039 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95f20b-eb79-4130-99a3-1712602aad21 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:38.653347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95f20b-eb79-4130-99a3-1712602aad21 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:38.653626 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95f20b-eb79-4130-99a3-1712602aad21 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:38.654374 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95f20b-eb79-4130-99a3-1712602aad21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:38.654681 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95f20b-eb79-4130-99a3-1712602aad21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:38.656128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95f20b-eb79-4130-99a3-1712602aad21 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:38.665072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 371/723] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:38.670560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a326c39-0ffb-4d45-b7b0-5b9ff114424f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:38.771222 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:38.784241 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a326c39-0ffb-4d45-b7b0-5b9ff114424f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:38.819798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfef12-9197-4e17-8ad4-e5216c297c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:38.820611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfef12-9197-4e17-8ad4-e5216c297c9a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:38.820890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfef12-9197-4e17-8ad4-e5216c297c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:38.821187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfef12-9197-4e17-8ad4-e5216c297c9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:38.833530 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfef12-9197-4e17-8ad4-e5216c297c9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:38.834990 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfef12-9197-4e17-8ad4-e5216c297c9a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:38.853186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-20cfef12-9197-4e17-8ad4-e5216c297c9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:38.853186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:38.853186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:38.853186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:38.853186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:38.853186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:38.853186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:38.853186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:38.853186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:38.853186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:38.853186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:38.853186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:38.853186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:38.853186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:38.853186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:38.854198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:38.855081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:38.855081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:38.855081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:38.855081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:38.855081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 353/724] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:38.860273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95f20b-eb79-4130-99a3-1712602aad21 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:38.865249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:38.995705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95f20b-eb79-4130-99a3-1712602aad21 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:39.041524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279226f-419c-43f3-a913-12a7d0c66312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:39.042108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279226f-419c-43f3-a913-12a7d0c66312 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:39.042389 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279226f-419c-43f3-a913-12a7d0c66312 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:39.042659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279226f-419c-43f3-a913-12a7d0c66312 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:39.043390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279226f-419c-43f3-a913-12a7d0c66312 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:39.043694 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279226f-419c-43f3-a913-12a7d0c66312 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:39.045065 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279226f-419c-43f3-a913-12a7d0c66312 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:39.060001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 372/725] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:39.065812 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfef12-9197-4e17-8ad4-e5216c297c9a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:39.176832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfef12-9197-4e17-8ad4-e5216c297c9a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:39.203019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19e494-fe5f-4fba-8f88-0936e3a92005 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:39.203503 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19e494-fe5f-4fba-8f88-0936e3a92005 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:39.203637 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19e494-fe5f-4fba-8f88-0936e3a92005 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:39.205047 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 373/726] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:17:39.210575 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f94e45-c48c-4663-9049-3674db926340 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:39.210863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f94e45-c48c-4663-9049-3674db926340 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:39.211144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f94e45-c48c-4663-9049-3674db926340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:39.227938 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669141a-ea35-4e71-b5d6-cf60ed444e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:39.228737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669141a-ea35-4e71-b5d6-cf60ed444e20 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:39.229013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669141a-ea35-4e71-b5d6-cf60ed444e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:39.229345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669141a-ea35-4e71-b5d6-cf60ed444e20 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:39.237494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669141a-ea35-4e71-b5d6-cf60ed444e20 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:39.239516 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:39.239977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669141a-ea35-4e71-b5d6-cf60ed444e20 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:39.240407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-d2f94e45-c48c-4663-9049-3674db926340 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:39.241856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d2f94e45-c48c-4663-9049-3674db926340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-6669141a-ea35-4e71-b5d6-cf60ed444e20 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:39.255889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:39.256865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:39.257888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:39.257888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:39.257888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:39.257888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:39.257888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:39.257888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:39.257888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:39.257888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:39.257888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:17:39.257888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:39.261534 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 354/727] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Jul 19 15:17:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:39.267176 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669141a-ea35-4e71-b5d6-cf60ed444e20 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:39.388181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669141a-ea35-4e71-b5d6-cf60ed444e20 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:39.406944 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:39.426309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a7d03-9cf2-4616-ae22-12e346fa06e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:39.426992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a7d03-9cf2-4616-ae22-12e346fa06e3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:39.427279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a7d03-9cf2-4616-ae22-12e346fa06e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:39.427556 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a7d03-9cf2-4616-ae22-12e346fa06e3 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:39.428426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a7d03-9cf2-4616-ae22-12e346fa06e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:39.428776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a7d03-9cf2-4616-ae22-12e346fa06e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:39.428934 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 374/728] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:39.430856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a7d03-9cf2-4616-ae22-12e346fa06e3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:39.453389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 355/729] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Jul 19 15:17:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:39.454816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f94e45-c48c-4663-9049-3674db926340 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:39.474075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a7d03-9cf2-4616-ae22-12e346fa06e3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:39.537290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:39.582815 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f94e45-c48c-4663-9049-3674db926340 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:39.592649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a7d03-9cf2-4616-ae22-12e346fa06e3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:39.618566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a3d6e-7d91-4365-a519-c30ce02f0b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:39.619647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a3d6e-7d91-4365-a519-c30ce02f0b54 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:39.620091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a3d6e-7d91-4365-a519-c30ce02f0b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:39.620515 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a3d6e-7d91-4365-a519-c30ce02f0b54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:39.631245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702f6f7-da7f-47f8-a2bb-26acf57930ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:39.632030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702f6f7-da7f-47f8-a2bb-26acf57930ae None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:39.632311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702f6f7-da7f-47f8-a2bb-26acf57930ae None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:39.632586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702f6f7-da7f-47f8-a2bb-26acf57930ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:39.632829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a3d6e-7d91-4365-a519-c30ce02f0b54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:39.634470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a3d6e-7d91-4365-a519-c30ce02f0b54 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:39.644811 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702f6f7-da7f-47f8-a2bb-26acf57930ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f702f6f7-da7f-47f8-a2bb-26acf57930ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:39.652680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:39.653803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:39.654809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:39.654809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:39.654809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:39.654809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:39.654809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:39.654809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:39.654809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:39.654809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:39.654809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:39.654809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:39.654809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:39.654809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:39.654809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 356/730] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:39.655538 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b806f7-37c3-4b3a-a9d7-582ab321ca62 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:39.655831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b806f7-37c3-4b3a-a9d7-582ab321ca62 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:39.656106 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b806f7-37c3-4b3a-a9d7-582ab321ca62 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:39.657591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-714a3d6e-7d91-4365-a519-c30ce02f0b54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:39.657591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:39.657591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:39.657591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:39.657591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:39.657591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:39.657591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:39.657591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:39.657591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:39.657591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:39.657591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:39.657591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:39.657591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:39.657591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:39.657591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:39.659144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:39.660464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:39.660464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:39.660464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:39.660464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:39.660748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 357/731] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:39.664911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 375/732] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:39.665055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b806f7-37c3-4b3a-a9d7-582ab321ca62 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:39.666593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98567c6e-d91b-453d-a9ad-82bf8000e53c None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:39.666870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98567c6e-d91b-453d-a9ad-82bf8000e53c None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:39.667139 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98567c6e-d91b-453d-a9ad-82bf8000e53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:39.694945 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-98567c6e-d91b-453d-a9ad-82bf8000e53c None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:39.696650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-98567c6e-d91b-453d-a9ad-82bf8000e53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:39.750881 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:39.793162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 376/733] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:39.793162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98567c6e-d91b-453d-a9ad-82bf8000e53c None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:39.805820 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b806f7-37c3-4b3a-a9d7-582ab321ca62 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:39.828396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:39.851618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2535732-6f62-456a-ab1a-639baccecaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:39.852660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2535732-6f62-456a-ab1a-639baccecaaa None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:39.853178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2535732-6f62-456a-ab1a-639baccecaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:39.853668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2535732-6f62-456a-ab1a-639baccecaaa None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:39.854623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2535732-6f62-456a-ab1a-639baccecaaa None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:39.855156 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2535732-6f62-456a-ab1a-639baccecaaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:39.856710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2535732-6f62-456a-ab1a-639baccecaaa None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:39.865923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 358/734] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:39.867153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2535732-6f62-456a-ab1a-639baccecaaa None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:39.899234 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98567c6e-d91b-453d-a9ad-82bf8000e53c None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:39.937274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdee206-e0a7-45f8-ab59-4750a7968c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:39.937995 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdee206-e0a7-45f8-ab59-4750a7968c31 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:39.938209 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdee206-e0a7-45f8-ab59-4750a7968c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:39.938484 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdee206-e0a7-45f8-ab59-4750a7968c31 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:39.939222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdee206-e0a7-45f8-ab59-4750a7968c31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:39.939533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdee206-e0a7-45f8-ab59-4750a7968c31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:39.942123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdee206-e0a7-45f8-ab59-4750a7968c31 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:39.958716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 377/735] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:39.958716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bd4dc-ccb8-4ba7-90f6-7f2a79adbb5f None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:39.958716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bd4dc-ccb8-4ba7-90f6-7f2a79adbb5f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:39.958716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bd4dc-ccb8-4ba7-90f6-7f2a79adbb5f None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:39.958716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 378/736] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:39.959141 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bd4dc-ccb8-4ba7-90f6-7f2a79adbb5f None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:39.975015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2535732-6f62-456a-ab1a-639baccecaaa None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:39.997067 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:40.009750 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75063ca4-ef29-4ce3-8cce-92f9c63665cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:40.010425 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75063ca4-ef29-4ce3-8cce-92f9c63665cd None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:40.010711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75063ca4-ef29-4ce3-8cce-92f9c63665cd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:40.010988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75063ca4-ef29-4ce3-8cce-92f9c63665cd None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:40.011703 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75063ca4-ef29-4ce3-8cce-92f9c63665cd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:40.013070 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75063ca4-ef29-4ce3-8cce-92f9c63665cd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:40.027514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 359/737] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Jul 19 15:17:39 2022] POST /identity/v3/services => generated 251 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:17:40.028497 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75063ca4-ef29-4ce3-8cce-92f9c63665cd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:40.076401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bd4dc-ccb8-4ba7-90f6-7f2a79adbb5f None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:40.076618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:40.116586 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc93497-0bd8-42a2-9abd-43f975f7e77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:40.117471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc93497-0bd8-42a2-9abd-43f975f7e77c None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:40.117753 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc93497-0bd8-42a2-9abd-43f975f7e77c None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:40.118026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc93497-0bd8-42a2-9abd-43f975f7e77c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:40.130056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc93497-0bd8-42a2-9abd-43f975f7e77c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:40.131627 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc93497-0bd8-42a2-9abd-43f975f7e77c None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:40.150387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-1dc93497-0bd8-42a2-9abd-43f975f7e77c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:40.150387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:40.150387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:40.150387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:40.150387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:40.150387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:40.150387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:40.150387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:40.150387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:40.150387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:40.150387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:40.150387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:40.150387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:40.150387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:40.150387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:40.151576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:40.152466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:40.152466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:40.152466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:40.152466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:40.152466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 379/738] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:40.153222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96e4be-1944-4e9f-9340-559330f1ebab None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:40.153401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96e4be-1944-4e9f-9340-559330f1ebab None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:40.153679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96e4be-1944-4e9f-9340-559330f1ebab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:40.167799 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75063ca4-ef29-4ce3-8cce-92f9c63665cd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:40.170653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-dd96e4be-1944-4e9f-9340-559330f1ebab None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:40.171856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-dd96e4be-1944-4e9f-9340-559330f1ebab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:40.218513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7aec59-114e-485a-b7e6-152a47cb8e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:40.219182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7aec59-114e-485a-b7e6-152a47cb8e93 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:40.219476 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7aec59-114e-485a-b7e6-152a47cb8e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:40.219761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7aec59-114e-485a-b7e6-152a47cb8e93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:40.227955 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7aec59-114e-485a-b7e6-152a47cb8e93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:40.229458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7aec59-114e-485a-b7e6-152a47cb8e93 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:40.244364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-dd7aec59-114e-485a-b7e6-152a47cb8e93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:40.244364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:40.244364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:40.244364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:40.244364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:40.244364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:40.244364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:40.244364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:40.244364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:40.244364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:40.244364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:40.244364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:40.244364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:40.244364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:40.244364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:40.245759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:40.247033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:40.247033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:40.247033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:40.247033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:40.247033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:40.247033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:40.247033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:40.247033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:40.247692 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 380/739] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:40 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:40.248083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 360/740] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:40.250702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e70d9-69e4-46c4-ae4d-f8ba2848d70d None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:40.250999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e70d9-69e4-46c4-ae4d-f8ba2848d70d None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:40.251281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e70d9-69e4-46c4-ae4d-f8ba2848d70d None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:40.253374 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7aec59-114e-485a-b7e6-152a47cb8e93 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:40.257172 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 381/741] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:40.257172 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e70d9-69e4-46c4-ae4d-f8ba2848d70d None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:40.274672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:40.360885 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:40.368311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7aec59-114e-485a-b7e6-152a47cb8e93 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:40.373390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e70d9-69e4-46c4-ae4d-f8ba2848d70d None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:40.408978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bc8f3-7b3c-4454-95fa-a6ed5e449f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:40.409689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bc8f3-7b3c-4454-95fa-a6ed5e449f79 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:40.409977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bc8f3-7b3c-4454-95fa-a6ed5e449f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:40.410253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bc8f3-7b3c-4454-95fa-a6ed5e449f79 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:40.410978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bc8f3-7b3c-4454-95fa-a6ed5e449f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:40.411286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bc8f3-7b3c-4454-95fa-a6ed5e449f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:40.412650 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bc8f3-7b3c-4454-95fa-a6ed5e449f79 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:40.413081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a2704-88a0-4cbd-a9ee-107e6f593630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:40.413763 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a2704-88a0-4cbd-a9ee-107e6f593630 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:40.414049 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a2704-88a0-4cbd-a9ee-107e6f593630 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:40.414308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a2704-88a0-4cbd-a9ee-107e6f593630 None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:40.415100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a2704-88a0-4cbd-a9ee-107e6f593630 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:40.415406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a2704-88a0-4cbd-a9ee-107e6f593630 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:40.416829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a2704-88a0-4cbd-a9ee-107e6f593630 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:40.429579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 361/742] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:40.429579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf4687-fd0d-45b7-a2a5-97a073f89d1f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:40.429579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf4687-fd0d-45b7-a2a5-97a073f89d1f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:40.429579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf4687-fd0d-45b7-a2a5-97a073f89d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:40.430027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 382/743] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:40.430396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a2704-88a0-4cbd-a9ee-107e6f593630 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:40.450908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c3cf4687-fd0d-45b7-a2a5-97a073f89d1f None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:40.452166 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c3cf4687-fd0d-45b7-a2a5-97a073f89d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:40.516369 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:40.526971 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 362/744] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:40 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 15:17:40.528201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf4687-fd0d-45b7-a2a5-97a073f89d1f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:40.546722 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a2704-88a0-4cbd-a9ee-107e6f593630 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:40.583189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e3a9b-d9e2-4ecd-91e0-a1cda1302ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:40.584116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e3a9b-d9e2-4ecd-91e0-a1cda1302ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:40.584301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e3a9b-d9e2-4ecd-91e0-a1cda1302ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:40.584531 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e3a9b-d9e2-4ecd-91e0-a1cda1302ae0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:40.595529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e3a9b-d9e2-4ecd-91e0-a1cda1302ae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-656e3a9b-d9e2-4ecd-91e0-a1cda1302ae0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:40.603322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:40.604268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:40.605115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:40.605115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:40.605115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:40.605115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:40.605115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:40.605115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:40.605115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:40.605115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:40.605115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:40.605115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:40.605115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:40.605115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:40.605115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 383/745] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:17:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:40.606500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e3a9b-d9e2-4ecd-91e0-a1cda1302ae0 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:40.626769 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:40.646359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf4687-fd0d-45b7-a2a5-97a073f89d1f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:40.682632 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc6756-88b2-4515-a3cf-64f84c8dffb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:40.683374 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc6756-88b2-4515-a3cf-64f84c8dffb4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:40.683658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc6756-88b2-4515-a3cf-64f84c8dffb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:40.683936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc6756-88b2-4515-a3cf-64f84c8dffb4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:40.692005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc6756-88b2-4515-a3cf-64f84c8dffb4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:40.693474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc6756-88b2-4515-a3cf-64f84c8dffb4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-7dcc6756-88b2-4515-a3cf-64f84c8dffb4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:40.708690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:40.709808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:40.710930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:40.710930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:40.710930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:40.710930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:40.710930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:40.710930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:40.710930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:40.710930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:40.710930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 15:17:40.710930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:40.710930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 363/746] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Jul 19 15:17:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:40.710930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc6756-88b2-4515-a3cf-64f84c8dffb4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:40.722336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e3a9b-d9e2-4ecd-91e0-a1cda1302ae0 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:40.758471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36baff2-60cd-4147-8cc4-a18a010f3b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:40.759523 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36baff2-60cd-4147-8cc4-a18a010f3b10 None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:40.759866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36baff2-60cd-4147-8cc4-a18a010f3b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:40.760143 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36baff2-60cd-4147-8cc4-a18a010f3b10 None admin] PATH_INFO: `/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/bff6a2d4bc734904b2615608a86b097d/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:40.762154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36baff2-60cd-4147-8cc4-a18a010f3b10 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5f51e47b8e34bf5ac822d8c93853e30, user_id=bff6a2d4bc734904b2615608a86b097d, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:40.763597 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36baff2-60cd-4147-8cc4-a18a010f3b10 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:40.766137 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:40.789173 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 384/747] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:17:40 2022] PUT /identity/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/bff6a2d4bc734904b2615608a86b097d/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:17:40.789173 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36baff2-60cd-4147-8cc4-a18a010f3b10 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:40.833507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc6756-88b2-4515-a3cf-64f84c8dffb4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:40.876092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7ffd8-f420-4b5d-a00e-6733310846a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:40.876903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7ffd8-f420-4b5d-a00e-6733310846a8 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:40.877915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7ffd8-f420-4b5d-a00e-6733310846a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:40.878345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7ffd8-f420-4b5d-a00e-6733310846a8 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:40.879176 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7ffd8-f420-4b5d-a00e-6733310846a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:40.881093 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7ffd8-f420-4b5d-a00e-6733310846a8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:40.904020 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 364/748] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Jul 19 15:17:40 2022] POST /identity/v3/services => generated 240 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:17:40.905168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7ffd8-f420-4b5d-a00e-6733310846a8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:40.916783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:40.931696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36baff2-60cd-4147-8cc4-a18a010f3b10 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:40.999341 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c3806-bf12-4e46-92ae-3a8371ed8f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:41.000026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c3806-bf12-4e46-92ae-3a8371ed8f2d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:41.000566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c3806-bf12-4e46-92ae-3a8371ed8f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:41.000849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c3806-bf12-4e46-92ae-3a8371ed8f2d None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:41.002046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c3806-bf12-4e46-92ae-3a8371ed8f2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:41.003226 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c3806-bf12-4e46-92ae-3a8371ed8f2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:41.004661 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c3806-bf12-4e46-92ae-3a8371ed8f2d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:41.023006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 385/749] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:41.030026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c3806-bf12-4e46-92ae-3a8371ed8f2d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:41.082346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7ffd8-f420-4b5d-a00e-6733310846a8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:41.098689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:41.128527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54889e-fce4-4b45-8d79-3be7c0d9e33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:41.129738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54889e-fce4-4b45-8d79-3be7c0d9e33b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:41.129738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54889e-fce4-4b45-8d79-3be7c0d9e33b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:41.129912 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54889e-fce4-4b45-8d79-3be7c0d9e33b None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:41.130644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54889e-fce4-4b45-8d79-3be7c0d9e33b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:41.130949 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54889e-fce4-4b45-8d79-3be7c0d9e33b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:41.132927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54889e-fce4-4b45-8d79-3be7c0d9e33b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:41.152971 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 365/750] 10.222.0.42 () {56 vars in 1149 bytes} [Tue Jul 19 15:17:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:41.159175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54889e-fce4-4b45-8d79-3be7c0d9e33b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:41.174001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c3806-bf12-4e46-92ae-3a8371ed8f2d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:41.224986 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d3441-9bf5-4823-9c7d-29071d96c39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:41.225856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d3441-9bf5-4823-9c7d-29071d96c39d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:41.226140 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d3441-9bf5-4823-9c7d-29071d96c39d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:41.226420 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d3441-9bf5-4823-9c7d-29071d96c39d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:41.240972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:41.244123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d3441-9bf5-4823-9c7d-29071d96c39d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:41.246118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d3441-9bf5-4823-9c7d-29071d96c39d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:41.270696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-7d1d3441-9bf5-4823-9c7d-29071d96c39d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:41.270696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:41.270696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:41.270696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:41.270696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:41.270696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:41.270696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:41.270696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:41.270696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:41.270696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:41.270696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:41.270696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:41.270696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:41.270696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:41.270696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:41.272074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:41.273632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:41.273632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:41.273632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:41.273632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:41.273632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 386/751] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:41.278991 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d3441-9bf5-4823-9c7d-29071d96c39d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:41.297316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54889e-fce4-4b45-8d79-3be7c0d9e33b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:41.338011 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c464f-2242-4ca2-8f40-0db970376c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:41.339112 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c464f-2242-4ca2-8f40-0db970376c63 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:41.339742 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c464f-2242-4ca2-8f40-0db970376c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:41.340222 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c464f-2242-4ca2-8f40-0db970376c63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:41.390248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d3441-9bf5-4823-9c7d-29071d96c39d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:41.392620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:41.430564 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c1747-93a3-4607-b5e1-088ac262134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:41.431241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c1747-93a3-4607-b5e1-088ac262134a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:41.431529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c1747-93a3-4607-b5e1-088ac262134a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:41.431810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c1747-93a3-4607-b5e1-088ac262134a None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:41.432935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c1747-93a3-4607-b5e1-088ac262134a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:41.432935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c1747-93a3-4607-b5e1-088ac262134a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:41.434281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c1747-93a3-4607-b5e1-088ac262134a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:41.435517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c464f-2242-4ca2-8f40-0db970376c63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:41.442283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c464f-2242-4ca2-8f40-0db970376c63 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:41.448359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 387/752] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:41.453850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c1747-93a3-4607-b5e1-088ac262134a None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:41.514676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:41.559283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c1747-93a3-4607-b5e1-088ac262134a None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:41.563197 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 366/753] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:41 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 15:17:41.572272 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c464f-2242-4ca2-8f40-0db970376c63 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:41.600106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eda210-0a3c-4845-9a9e-16847c6526fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:41.600910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eda210-0a3c-4845-9a9e-16847c6526fc None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:41.605157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eda210-0a3c-4845-9a9e-16847c6526fc None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:41.605157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eda210-0a3c-4845-9a9e-16847c6526fc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:41.608971 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eda210-0a3c-4845-9a9e-16847c6526fc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:41.610687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eda210-0a3c-4845-9a9e-16847c6526fc None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-92eda210-0a3c-4845-9a9e-16847c6526fc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:41.625471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:41.626431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:41.628888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:41.628888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:41.628888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:41.628888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:41.628888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:41.628888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:41.628888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:41.628888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:41.628888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 15:17:41.628888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:41.628888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 388/754] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:17:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:41.633510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eda210-0a3c-4845-9a9e-16847c6526fc None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:41.681333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:41.742202 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c464f-2242-4ca2-8f40-0db970376c63 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:41.755403 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eda210-0a3c-4845-9a9e-16847c6526fc None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:41.786015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d950e-3f80-438c-83f3-517ef87284b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:41.787980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d950e-3f80-438c-83f3-517ef87284b9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:41.788627 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d950e-3f80-438c-83f3-517ef87284b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:41.790355 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d950e-3f80-438c-83f3-517ef87284b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:41.805254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d950e-3f80-438c-83f3-517ef87284b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:41.807291 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d950e-3f80-438c-83f3-517ef87284b9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:41.826607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c29d950e-3f80-438c-83f3-517ef87284b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:41.826607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:41.826607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:41.826607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:41.826607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:41.826607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:41.826607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:41.826607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:41.826607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:41.826607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:41.826607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:41.826607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:41.826607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:41.826607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:41.826607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:41.827539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:41.828413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:41.828413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:41.828413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:41.828413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:41.830894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 367/755] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:41.837947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfbd1d-3de4-41d7-9976-ca3dc68dd964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:41.838649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfbd1d-3de4-41d7-9976-ca3dc68dd964 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:41.838938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfbd1d-3de4-41d7-9976-ca3dc68dd964 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:41.839214 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfbd1d-3de4-41d7-9976-ca3dc68dd964 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:41.839732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d950e-3f80-438c-83f3-517ef87284b9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:41.840085 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfbd1d-3de4-41d7-9976-ca3dc68dd964 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:41.840402 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfbd1d-3de4-41d7-9976-ca3dc68dd964 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:41.841895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfbd1d-3de4-41d7-9976-ca3dc68dd964 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:41.862222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 389/756] 10.222.0.42 () {56 vars in 1135 bytes} [Tue Jul 19 15:17:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:41.867538 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfbd1d-3de4-41d7-9976-ca3dc68dd964 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:41.897464 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:41.953120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:41.973257 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfbd1d-3de4-41d7-9976-ca3dc68dd964 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:42.010831 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a87b65-91ac-46ba-b775-de51a452b521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:42.011492 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a87b65-91ac-46ba-b775-de51a452b521 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:42.011800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a87b65-91ac-46ba-b775-de51a452b521 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:42.012064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a87b65-91ac-46ba-b775-de51a452b521 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:42.053815 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d950e-3f80-438c-83f3-517ef87284b9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:42.112493 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a87b65-91ac-46ba-b775-de51a452b521 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:42.114117 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a87b65-91ac-46ba-b775-de51a452b521 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:42.157100 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c0f02-e0ab-48fc-99b8-659539327cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:42.157797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c0f02-e0ab-48fc-99b8-659539327cec None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:42.158086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c0f02-e0ab-48fc-99b8-659539327cec None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:42.159001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c0f02-e0ab-48fc-99b8-659539327cec None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:42.159743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c0f02-e0ab-48fc-99b8-659539327cec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:42.160056 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c0f02-e0ab-48fc-99b8-659539327cec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:42.161475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c0f02-e0ab-48fc-99b8-659539327cec None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:42.180991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 368/757] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:42.187010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c0f02-e0ab-48fc-99b8-659539327cec None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:42.259396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:42.296596 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 390/758] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:41 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 15:17:42.303473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a87b65-91ac-46ba-b775-de51a452b521 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:42.325589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:42.366155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c0f02-e0ab-48fc-99b8-659539327cec None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:42.453155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fec0d-eade-42ef-85af-006cf4df2ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:42.454011 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fec0d-eade-42ef-85af-006cf4df2ace None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:42.454304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fec0d-eade-42ef-85af-006cf4df2ace None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:42.454588 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fec0d-eade-42ef-85af-006cf4df2ace None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:42.468603 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fec0d-eade-42ef-85af-006cf4df2ace None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:42.470555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fec0d-eade-42ef-85af-006cf4df2ace None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:42.499947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-292fec0d-eade-42ef-85af-006cf4df2ace None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:42.499947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:42.499947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:42.499947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:42.499947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:42.499947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:42.499947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:42.499947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:42.499947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:42.499947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:42.499947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:42.499947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:42.499947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:42.499947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:42.499947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:42.500883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:42.503004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:42.503004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:42.503004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:42.503004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:42.503004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:42.503004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:42.503004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:42.503004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:42.503004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 369/759] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:42.508491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fec0d-eade-42ef-85af-006cf4df2ace None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:42.517497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a87b65-91ac-46ba-b775-de51a452b521 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:42.633056 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fec0d-eade-42ef-85af-006cf4df2ace None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:42.672541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c96ee6-423c-439e-98cf-709436d1067f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:42.673602 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c96ee6-423c-439e-98cf-709436d1067f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:42.674169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c96ee6-423c-439e-98cf-709436d1067f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:42.674644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c96ee6-423c-439e-98cf-709436d1067f None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:42.675756 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c96ee6-423c-439e-98cf-709436d1067f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:42.676371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c96ee6-423c-439e-98cf-709436d1067f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:42.678016 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c96ee6-423c-439e-98cf-709436d1067f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:42.681045 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:42.694446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 370/760] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:42.696143 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26a8b8-817a-4a12-b04f-694e2f20b9a8 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:42.696679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26a8b8-817a-4a12-b04f-694e2f20b9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:42.697207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26a8b8-817a-4a12-b04f-694e2f20b9a8 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:42.702677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23193269-b237-485d-921f-34f821addf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:42.702677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23193269-b237-485d-921f-34f821addf89 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:42.702677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23193269-b237-485d-921f-34f821addf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:42.702677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23193269-b237-485d-921f-34f821addf89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:42.703235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 371/761] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:42.704274 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26a8b8-817a-4a12-b04f-694e2f20b9a8 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:42.715334 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23193269-b237-485d-921f-34f821addf89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:42.717451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23193269-b237-485d-921f-34f821addf89 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:42.738008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-23193269-b237-485d-921f-34f821addf89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:42.738008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:42.738008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:42.738008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:42.738008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:42.738008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:42.738008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:42.738008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:42.738008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:42.738008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:42.738008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:42.738008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:42.738008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:42.738008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:42.738008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:42.738965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:42.740318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:42.740318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:42.740318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:42.740318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:42.740318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 391/762] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:42.740907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140e665-05cc-4461-99dc-1d2d2a69286b None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:42.741183 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140e665-05cc-4461-99dc-1d2d2a69286b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:42.741459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140e665-05cc-4461-99dc-1d2d2a69286b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:42.761173 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-3140e665-05cc-4461-99dc-1d2d2a69286b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:42.762938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3140e665-05cc-4461-99dc-1d2d2a69286b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:42.800044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:42.834954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26a8b8-817a-4a12-b04f-694e2f20b9a8 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:42.856270 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 392/763] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:17:42.857578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140e665-05cc-4461-99dc-1d2d2a69286b None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:42.872852 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484afb07-9c90-478d-8983-6033d2415c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:42.873718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484afb07-9c90-478d-8983-6033d2415c61 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:42.873845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484afb07-9c90-478d-8983-6033d2415c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:42.874096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484afb07-9c90-478d-8983-6033d2415c61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:42.874938 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484afb07-9c90-478d-8983-6033d2415c61 None admin] RBAC: Adding query filter params (role.id=c8578ede4e7041ecaeba79c255cc650b, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=60f89f6042664f91b2829d516ed7e844) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:42.875253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484afb07-9c90-478d-8983-6033d2415c61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:42.877075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484afb07-9c90-478d-8983-6033d2415c61 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:42.883573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 372/764] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:42 2022] GET /identity/v3/role_assignments?user.id=60f89f6042664f91b2829d516ed7e844&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=c8578ede4e7041ecaeba79c255cc650b => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:42.896387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484afb07-9c90-478d-8983-6033d2415c61 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:42.972725 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:43.034955 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484afb07-9c90-478d-8983-6033d2415c61 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:43.053264 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140e665-05cc-4461-99dc-1d2d2a69286b None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:43.094636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ac53d-1e76-41d0-aae7-0a2d450aadcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:43.095314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ac53d-1e76-41d0-aae7-0a2d450aadcc None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:43.095595 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ac53d-1e76-41d0-aae7-0a2d450aadcc None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:43.096463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ac53d-1e76-41d0-aae7-0a2d450aadcc None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:43.097243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ac53d-1e76-41d0-aae7-0a2d450aadcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:43.097550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ac53d-1e76-41d0-aae7-0a2d450aadcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:43.098918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ac53d-1e76-41d0-aae7-0a2d450aadcc None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:43.102989 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114a5d4-e5f4-47c3-ba2a-b9ddffc582e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:43.103808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114a5d4-e5f4-47c3-ba2a-b9ddffc582e7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:43.104082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114a5d4-e5f4-47c3-ba2a-b9ddffc582e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:43.104354 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114a5d4-e5f4-47c3-ba2a-b9ddffc582e7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:43.105019 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114a5d4-e5f4-47c3-ba2a-b9ddffc582e7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:43.106391 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114a5d4-e5f4-47c3-ba2a-b9ddffc582e7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c114a5d4-e5f4-47c3-ba2a-b9ddffc582e7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:43.126084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:43.127196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:43.128036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:43.128036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:43.128036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:43.128036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:43.128036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:43.128036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:43.128036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:43.128036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:43.128036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:43.128036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:43.128036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:43.128036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:17:43.128036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:43.128036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 15:17:43.128036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:43.136240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 393/765] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:43.136484 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 373/766] 10.222.0.42 () {56 vars in 1129 bytes} [Tue Jul 19 15:17:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:43.138321 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ac53d-1e76-41d0-aae7-0a2d450aadcc None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:43.142398 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114a5d4-e5f4-47c3-ba2a-b9ddffc582e7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:43.185529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:43.267249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114a5d4-e5f4-47c3-ba2a-b9ddffc582e7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:43.293112 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ac53d-1e76-41d0-aae7-0a2d450aadcc None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:43.302350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:43.319921 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53e8bd-8996-42b1-b6cf-d1c0577a5aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:43.321119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53e8bd-8996-42b1-b6cf-d1c0577a5aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:43.321869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53e8bd-8996-42b1-b6cf-d1c0577a5aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:43.322744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53e8bd-8996-42b1-b6cf-d1c0577a5aa9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:43.331291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4dacec-b724-461d-94fe-4839ea6bb605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:43.332034 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dacec-b724-461d-94fe-4839ea6bb605 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:43.332386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dacec-b724-461d-94fe-4839ea6bb605 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:43.332739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dacec-b724-461d-94fe-4839ea6bb605 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:43.333559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4dacec-b724-461d-94fe-4839ea6bb605 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:43.333934 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4dacec-b724-461d-94fe-4839ea6bb605 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:43.334438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53e8bd-8996-42b1-b6cf-d1c0577a5aa9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:43.335457 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4dacec-b724-461d-94fe-4839ea6bb605 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:43.336804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53e8bd-8996-42b1-b6cf-d1c0577a5aa9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:43.340685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 394/767] 10.222.0.42 () {56 vars in 1124 bytes} [Tue Jul 19 15:17:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:43.342317 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9cc60-b71b-4465-be34-f74027bccd64 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:43.342675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9cc60-b71b-4465-be34-f74027bccd64 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:43.343025 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9cc60-b71b-4465-be34-f74027bccd64 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:43.348715 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 395/768] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:43.349716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9cc60-b71b-4465-be34-f74027bccd64 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:43.355835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9b53e8bd-8996-42b1-b6cf-d1c0577a5aa9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:43.355835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:43.355835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:43.355835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:43.355835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:43.355835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:43.355835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:43.355835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:43.355835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:43.355835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:43.355835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:43.355835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:43.355835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:43.355835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:43.355835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:43.356752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:43.357712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:43.357712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:43.357712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:43.357712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:43.357712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:43.357712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:43.357712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:43.357712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:43.360851 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 374/769] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:43.366552 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fe051-bb3f-43bd-9198-dce07fa61ca8 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:43.366840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fe051-bb3f-43bd-9198-dce07fa61ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:43.367116 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fe051-bb3f-43bd-9198-dce07fa61ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:43.394259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ad5fe051-bb3f-43bd-9198-dce07fa61ca8 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:43.396074 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ad5fe051-bb3f-43bd-9198-dce07fa61ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:43.460662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:43.474187 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9cc60-b71b-4465-be34-f74027bccd64 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:43.475248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 375/770] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:17:43.475935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fe051-bb3f-43bd-9198-dce07fa61ca8 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:43.530735 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d4c11-10de-4e56-ba90-cb56b4d4f0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:43.531516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d4c11-10de-4e56-ba90-cb56b4d4f0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:43.531804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d4c11-10de-4e56-ba90-cb56b4d4f0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:43.532086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d4c11-10de-4e56-ba90-cb56b4d4f0c2 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:43.532814 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d4c11-10de-4e56-ba90-cb56b4d4f0c2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:43.533124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d4c11-10de-4e56-ba90-cb56b4d4f0c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:43.534614 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d4c11-10de-4e56-ba90-cb56b4d4f0c2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:43.540473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 396/771] 10.222.0.42 () {56 vars in 1124 bytes} [Tue Jul 19 15:17:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:43.541795 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a963d-e4ba-40a2-ba01-97c3e9b22305 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:43.542141 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a963d-e4ba-40a2-ba01-97c3e9b22305 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:43.542420 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a963d-e4ba-40a2-ba01-97c3e9b22305 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:43.547825 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 397/772] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:43.547825 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a963d-e4ba-40a2-ba01-97c3e9b22305 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:43.596103 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fe051-bb3f-43bd-9198-dce07fa61ca8 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:43.612275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:43.637457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ab119-f174-40a0-bf56-0d5765655bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:43.638127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ab119-f174-40a0-bf56-0d5765655bfb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:43.638418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ab119-f174-40a0-bf56-0d5765655bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:43.638695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ab119-f174-40a0-bf56-0d5765655bfb None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:43.639489 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ab119-f174-40a0-bf56-0d5765655bfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:43.639801 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ab119-f174-40a0-bf56-0d5765655bfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:43.641362 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ab119-f174-40a0-bf56-0d5765655bfb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:43.649751 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a963d-e4ba-40a2-ba01-97c3e9b22305 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:43.659524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 376/773] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:43.659524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ab119-f174-40a0-bf56-0d5765655bfb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:43.687601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e20d0a-faac-484f-befc-8bd6d5290387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:43.688706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e20d0a-faac-484f-befc-8bd6d5290387 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:43.689207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e20d0a-faac-484f-befc-8bd6d5290387 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:43.690245 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e20d0a-faac-484f-befc-8bd6d5290387 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:43.701234 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e20d0a-faac-484f-befc-8bd6d5290387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-d6e20d0a-faac-484f-befc-8bd6d5290387 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:43.708963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:43.710059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:43.710932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:43.710932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:43.710932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:43.710932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:43.710932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:43.710932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:43.710932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:43.710932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:43.710932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:43.710932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:43.710932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:43.710932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:43.710932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 398/774] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:17:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:43.711661 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e09ae-81eb-471f-917b-4a39bb500970 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:43.711741 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e09ae-81eb-471f-917b-4a39bb500970 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:43.712636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e09ae-81eb-471f-917b-4a39bb500970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:43.730010 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-615e09ae-81eb-471f-917b-4a39bb500970 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:43.731200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-615e09ae-81eb-471f-917b-4a39bb500970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:43.740407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:43.770294 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ab119-f174-40a0-bf56-0d5765655bfb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:43.804864 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1a1f4-6f2d-4d59-a36e-b38c53145c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:43.805635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1a1f4-6f2d-4d59-a36e-b38c53145c80 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:43.805915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1a1f4-6f2d-4d59-a36e-b38c53145c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:43.806189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1a1f4-6f2d-4d59-a36e-b38c53145c80 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:43.806980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1a1f4-6f2d-4d59-a36e-b38c53145c80 None admin] RBAC: Adding query filter params (interface=public, service_id=8e630bd5c19842f8964fd1c52798f2dc, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:43.807290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1a1f4-6f2d-4d59-a36e-b38c53145c80 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:43.808683 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 399/775] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:17:43.808796 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1a1f4-6f2d-4d59-a36e-b38c53145c80 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:43.809777 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e09ae-81eb-471f-917b-4a39bb500970 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:43.821658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 377/776] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Jul 19 15:17:43 2022] GET /identity/v3/endpoints?service_id=8e630bd5c19842f8964fd1c52798f2dc&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:43.821658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1a1f4-6f2d-4d59-a36e-b38c53145c80 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:43.860392 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:43.922004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1a1f4-6f2d-4d59-a36e-b38c53145c80 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:43.933713 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e09ae-81eb-471f-917b-4a39bb500970 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:43.956920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65decde2-bcc4-4c05-9468-dd6ffebd6a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:43.957626 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65decde2-bcc4-4c05-9468-dd6ffebd6a3c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:43.957912 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65decde2-bcc4-4c05-9468-dd6ffebd6a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:43.958190 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65decde2-bcc4-4c05-9468-dd6ffebd6a3c None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:43.958947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65decde2-bcc4-4c05-9468-dd6ffebd6a3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:43.959259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65decde2-bcc4-4c05-9468-dd6ffebd6a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:43.960667 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65decde2-bcc4-4c05-9468-dd6ffebd6a3c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:43.968948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 378/777] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:43.969570 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65decde2-bcc4-4c05-9468-dd6ffebd6a3c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:43.975360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b14ae6-8844-46cd-a4ee-c9d3714cd54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:43.976243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b14ae6-8844-46cd-a4ee-c9d3714cd54f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:43.976729 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b14ae6-8844-46cd-a4ee-c9d3714cd54f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:43.977198 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b14ae6-8844-46cd-a4ee-c9d3714cd54f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:43.978275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b14ae6-8844-46cd-a4ee-c9d3714cd54f None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=96faaac55bc5408685c68a6177d91971) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:43.978883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b14ae6-8844-46cd-a4ee-c9d3714cd54f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:43.980319 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b14ae6-8844-46cd-a4ee-c9d3714cd54f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:43.989443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 400/778] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:43 2022] GET /identity/v3/role_assignments?user.id=96faaac55bc5408685c68a6177d91971&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:43.990195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b14ae6-8844-46cd-a4ee-c9d3714cd54f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:44.003126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:44.103127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:44.107205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65decde2-bcc4-4c05-9468-dd6ffebd6a3c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:44.113958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b14ae6-8844-46cd-a4ee-c9d3714cd54f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:44.160325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae5a68-560a-4246-bb08-056e9c93568c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:44.160710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf2188-a4ee-4e0b-910b-9d1074bf02df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:44.161129 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae5a68-560a-4246-bb08-056e9c93568c None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:44.161448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae5a68-560a-4246-bb08-056e9c93568c None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:44.161727 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae5a68-560a-4246-bb08-056e9c93568c None admin] PATH_INFO: `/v3/services/network` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:44.162386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae5a68-560a-4246-bb08-056e9c93568c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:44.162756 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf2188-a4ee-4e0b-910b-9d1074bf02df None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:44.163229 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf2188-a4ee-4e0b-910b-9d1074bf02df None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:44.163706 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf2188-a4ee-4e0b-910b-9d1074bf02df None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:44.163887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae5a68-560a-4246-bb08-056e9c93568c None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:44.164762 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf2188-a4ee-4e0b-910b-9d1074bf02df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:44.166361 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf2188-a4ee-4e0b-910b-9d1074bf02df None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:44.172331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 379/779] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Jul 19 15:17:43 2022] GET /identity/v3/services => generated 772 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-34ae5a68-560a-4246-bb08-056e9c93568c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:44.177967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:44.179006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:44.179841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:44.179841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:44.179841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:44.179841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:44.179841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:44.179841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:44.179841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:44.179841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:44.179841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:44.179841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:44.179841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:17:44.179841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:44.179841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 15:17:44.179841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:44.179841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 401/780] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:43 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:44.180622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae5a68-560a-4246-bb08-056e9c93568c None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:44.185068 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf2188-a4ee-4e0b-910b-9d1074bf02df None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:44.266861 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:44.308169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae5a68-560a-4246-bb08-056e9c93568c None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:44.349432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf88bef-2d63-4029-81f8-fc2eed7ead8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:44.351451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf88bef-2d63-4029-81f8-fc2eed7ead8f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:44.351908 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf88bef-2d63-4029-81f8-fc2eed7ead8f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:44.352325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf88bef-2d63-4029-81f8-fc2eed7ead8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:44.365544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf88bef-2d63-4029-81f8-fc2eed7ead8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:44.367589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf88bef-2d63-4029-81f8-fc2eed7ead8f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:44.388832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-eaf88bef-2d63-4029-81f8-fc2eed7ead8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:44.388832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:44.388832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:44.388832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:44.388832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:44.388832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:44.388832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:44.388832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:44.388832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:44.388832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:44.388832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:44.388832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:44.388832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:44.388832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:44.388832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:44.389851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:44.390743 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:44.390743 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:44.390743 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:44.390743 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:44.394059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 402/781] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:44.404837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf88bef-2d63-4029-81f8-fc2eed7ead8f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:44.447182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf2188-a4ee-4e0b-910b-9d1074bf02df None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:44.504984 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64deb147-2e2f-4fbb-8341-2b37c857e452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:44.506985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64deb147-2e2f-4fbb-8341-2b37c857e452 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:44.507479 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64deb147-2e2f-4fbb-8341-2b37c857e452 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:44.507951 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64deb147-2e2f-4fbb-8341-2b37c857e452 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:44.508878 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64deb147-2e2f-4fbb-8341-2b37c857e452 None admin] RBAC: Adding query filter params (name=network) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:44.509424 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64deb147-2e2f-4fbb-8341-2b37c857e452 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:44.511497 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64deb147-2e2f-4fbb-8341-2b37c857e452 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:44.518838 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 380/782] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Jul 19 15:17:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:44.526357 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64deb147-2e2f-4fbb-8341-2b37c857e452 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:44.546332 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf88bef-2d63-4029-81f8-fc2eed7ead8f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:44.573517 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:44.573750 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:44.590016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d96135-183f-449d-a6d0-bbffc4a4f5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:44.590883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d96135-183f-449d-a6d0-bbffc4a4f5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:44.590996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d96135-183f-449d-a6d0-bbffc4a4f5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:44.591277 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d96135-183f-449d-a6d0-bbffc4a4f5f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:44.592017 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d96135-183f-449d-a6d0-bbffc4a4f5f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:44.592419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d96135-183f-449d-a6d0-bbffc4a4f5f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:44.593791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d96135-183f-449d-a6d0-bbffc4a4f5f1 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:44.607231 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 403/783] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:44.615763 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d96135-183f-449d-a6d0-bbffc4a4f5f1 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:44.660054 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64deb147-2e2f-4fbb-8341-2b37c857e452 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:44.701081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede2509-d419-4dcb-8125-20193063266d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:44.702228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede2509-d419-4dcb-8125-20193063266d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:44.702711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede2509-d419-4dcb-8125-20193063266d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:44.703184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede2509-d419-4dcb-8125-20193063266d None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:44.704372 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede2509-d419-4dcb-8125-20193063266d None admin] RBAC: Adding query filter params (type=network) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:44.704880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede2509-d419-4dcb-8125-20193063266d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:44.707571 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede2509-d419-4dcb-8125-20193063266d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:44.715828 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 381/784] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Jul 19 15:17:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:44.724365 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede2509-d419-4dcb-8125-20193063266d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:44.839739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d96135-183f-449d-a6d0-bbffc4a4f5f1 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:44.909577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28c0d1-99e2-44b4-8885-ca7d3bcf357b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:44.910376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28c0d1-99e2-44b4-8885-ca7d3bcf357b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:44.910659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28c0d1-99e2-44b4-8885-ca7d3bcf357b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:44.910934 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28c0d1-99e2-44b4-8885-ca7d3bcf357b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:44.912381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:44.928766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede2509-d419-4dcb-8125-20193063266d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:44.931393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28c0d1-99e2-44b4-8885-ca7d3bcf357b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:44.932859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28c0d1-99e2-44b4-8885-ca7d3bcf357b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-9b28c0d1-99e2-44b4-8885-ca7d3bcf357b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:44.964322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:44.965327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:44.966300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:44.966300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:44.966300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:44.966300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:44.966300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:44.966300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:44.966300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:44.966300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:44.966300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:17:44.966300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:44.966300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 404/785] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Jul 19 15:17:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:44.971994 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28c0d1-99e2-44b4-8885-ca7d3bcf357b None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:45.001124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d801b-313c-4f83-bb76-746c892e8f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:45.002710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d801b-313c-4f83-bb76-746c892e8f5d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:45.004168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d801b-313c-4f83-bb76-746c892e8f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:45.004816 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d801b-313c-4f83-bb76-746c892e8f5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:45.007011 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d801b-313c-4f83-bb76-746c892e8f5d None admin] RBAC: Adding query filter params (interface=public, service_id=99f45fd07d2048f69c6add4dacd4c376, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:45.007595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d801b-313c-4f83-bb76-746c892e8f5d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:45.009323 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d801b-313c-4f83-bb76-746c892e8f5d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:45.022326 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 382/786] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Jul 19 15:17:44 2022] GET /identity/v3/endpoints?service_id=99f45fd07d2048f69c6add4dacd4c376&interface=public®ion_id=RegionOne => generated 194 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:45.028147 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d801b-313c-4f83-bb76-746c892e8f5d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:45.059308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:45.137279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28c0d1-99e2-44b4-8885-ca7d3bcf357b None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:45.180783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d801b-313c-4f83-bb76-746c892e8f5d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:45.186394 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a1d40-bd02-4617-9415-bb09caadb1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:45.187084 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a1d40-bd02-4617-9415-bb09caadb1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:45.187367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a1d40-bd02-4617-9415-bb09caadb1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:45.187647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a1d40-bd02-4617-9415-bb09caadb1f6 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:45.188504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a1d40-bd02-4617-9415-bb09caadb1f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:45.188818 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a1d40-bd02-4617-9415-bb09caadb1f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:45.190342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a1d40-bd02-4617-9415-bb09caadb1f6 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:45.219667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 405/787] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Jul 19 15:17:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:45.219667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf250f-4239-4faf-ad6b-047859818cb9 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:45.219667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf250f-4239-4faf-ad6b-047859818cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:45.219667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf250f-4239-4faf-ad6b-047859818cb9 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:45.219667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 406/788] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:45.220620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf250f-4239-4faf-ad6b-047859818cb9 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:45.224652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97f9f1-8ada-46fe-84e7-7d861e71003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:45.225631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97f9f1-8ada-46fe-84e7-7d861e71003a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:45.226132 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97f9f1-8ada-46fe-84e7-7d861e71003a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:45.226601 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97f9f1-8ada-46fe-84e7-7d861e71003a None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:45.227496 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97f9f1-8ada-46fe-84e7-7d861e71003a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:45.229055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97f9f1-8ada-46fe-84e7-7d861e71003a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:45.235391 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 383/789] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Jul 19 15:17:45 2022] GET /identity/v3/services => generated 772 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:45.237410 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe55a27-ce7b-4eae-a3f0-ade8959653b4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:45.237923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe55a27-ce7b-4eae-a3f0-ade8959653b4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:45.238394 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe55a27-ce7b-4eae-a3f0-ade8959653b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:45.268435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2fe55a27-ce7b-4eae-a3f0-ade8959653b4 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:45.270099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2fe55a27-ce7b-4eae-a3f0-ade8959653b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:45.288664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:45.395082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 384/790] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:17:45.429270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe55a27-ce7b-4eae-a3f0-ade8959653b4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:45.439114 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:45.446205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf250f-4239-4faf-ad6b-047859818cb9 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:45.482867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433dc29-9d1d-4ff0-9846-f7ec6fd0a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:45.483518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433dc29-9d1d-4ff0-9846-f7ec6fd0a77b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:45.483801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433dc29-9d1d-4ff0-9846-f7ec6fd0a77b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:45.484072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433dc29-9d1d-4ff0-9846-f7ec6fd0a77b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:45.544916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe55a27-ce7b-4eae-a3f0-ade8959653b4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:45.554442 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433dc29-9d1d-4ff0-9846-f7ec6fd0a77b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:45.555907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433dc29-9d1d-4ff0-9846-f7ec6fd0a77b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:45.586150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d0aeb-c901-4c8e-949d-49048f464a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:45.587110 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d0aeb-c901-4c8e-949d-49048f464a59 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:45.587493 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d0aeb-c901-4c8e-949d-49048f464a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:45.587780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d0aeb-c901-4c8e-949d-49048f464a59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:45.599970 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d0aeb-c901-4c8e-949d-49048f464a59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-363d0aeb-c901-4c8e-949d-49048f464a59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:45.608288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:45.609259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:45.610099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:45.610099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:45.610099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:45.610099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:45.610099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:45.610099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:45.610099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:45.610099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:45.610099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:45.610099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:45.610099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:45.610099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:45.610099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 385/791] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:45.617466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d0aeb-c901-4c8e-949d-49048f464a59 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:45.639959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:45.663343 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 407/792] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:17:45.665521 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:45.669240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433dc29-9d1d-4ff0-9846-f7ec6fd0a77b None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:45.776419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433dc29-9d1d-4ff0-9846-f7ec6fd0a77b None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:45.811018 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf13bd-0e54-4a16-8930-d63f88ba7537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:45.811812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf13bd-0e54-4a16-8930-d63f88ba7537 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:45.812092 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf13bd-0e54-4a16-8930-d63f88ba7537 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:45.812365 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf13bd-0e54-4a16-8930-d63f88ba7537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:45.825065 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf13bd-0e54-4a16-8930-d63f88ba7537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:45.826644 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf13bd-0e54-4a16-8930-d63f88ba7537 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:45.835099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d0aeb-c901-4c8e-949d-49048f464a59 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:45.846784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-b2bf13bd-0e54-4a16-8930-d63f88ba7537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:45.846784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:45.846784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:45.846784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:45.846784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:45.846784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:45.846784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:45.846784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:45.846784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:45.846784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:45.846784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:45.846784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:45.846784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:45.846784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:45.846784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:45.848817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:45.851030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:45.851030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:45.851030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:45.851030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:45.851030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 408/793] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:45.854453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf13bd-0e54-4a16-8930-d63f88ba7537 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:45.930570 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:45.967053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf13bd-0e54-4a16-8930-d63f88ba7537 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:45.987530 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9bb2e-9bc7-438f-b32b-9036539d23e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:45.988260 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9bb2e-9bc7-438f-b32b-9036539d23e5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:45.988541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9bb2e-9bc7-438f-b32b-9036539d23e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:45.988814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9bb2e-9bc7-438f-b32b-9036539d23e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:45.989607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9bb2e-9bc7-438f-b32b-9036539d23e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:45.989974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9bb2e-9bc7-438f-b32b-9036539d23e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:45.991378 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9bb2e-9bc7-438f-b32b-9036539d23e5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:46.011027 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 386/794] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:46.012168 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb7b4c-cbfa-428c-b904-6e4d6e1dcaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:46.012963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb7b4c-cbfa-428c-b904-6e4d6e1dcaff None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:46.013269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb7b4c-cbfa-428c-b904-6e4d6e1dcaff None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:46.013541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb7b4c-cbfa-428c-b904-6e4d6e1dcaff None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:46.014273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb7b4c-cbfa-428c-b904-6e4d6e1dcaff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:46.014578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb7b4c-cbfa-428c-b904-6e4d6e1dcaff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:46.015955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb7b4c-cbfa-428c-b904-6e4d6e1dcaff None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:46.018417 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9bb2e-9bc7-438f-b32b-9036539d23e5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:46.036328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 409/795] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:46.042195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb7b4c-cbfa-428c-b904-6e4d6e1dcaff None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:46.112356 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:46.140089 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9bb2e-9bc7-438f-b32b-9036539d23e5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:46.178922 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb20cb-335b-4c7b-acdc-103aebd8d9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:46.179718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb20cb-335b-4c7b-acdc-103aebd8d9df None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:46.179997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb20cb-335b-4c7b-acdc-103aebd8d9df None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:46.180273 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb20cb-335b-4c7b-acdc-103aebd8d9df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:46.193184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb20cb-335b-4c7b-acdc-103aebd8d9df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:46.194650 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb20cb-335b-4c7b-acdc-103aebd8d9df None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:46.213767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-4fcb20cb-335b-4c7b-acdc-103aebd8d9df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:46.213767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:46.213767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:46.213767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:46.213767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:46.213767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:46.213767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:46.213767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:46.213767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:46.213767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:46.213767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:46.213767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:46.213767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:46.213767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:46.213767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:46.214719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:46.215616 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:46.215616 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:46.215616 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:46.215616 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:46.215841 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 387/796] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:46.221616 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb20cb-335b-4c7b-acdc-103aebd8d9df None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:46.269015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb7b4c-cbfa-428c-b904-6e4d6e1dcaff None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:46.286637 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:46.321033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4626434-04cf-4a37-8496-4216ca463484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:46.322402 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4626434-04cf-4a37-8496-4216ca463484 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:46.323102 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4626434-04cf-4a37-8496-4216ca463484 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:46.323526 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4626434-04cf-4a37-8496-4216ca463484 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:46.332299 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4626434-04cf-4a37-8496-4216ca463484 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:46.333963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4626434-04cf-4a37-8496-4216ca463484 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:46.348224 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb20cb-335b-4c7b-acdc-103aebd8d9df None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:46.353366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-a4626434-04cf-4a37-8496-4216ca463484 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:46.353366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:46.353366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:46.353366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:46.353366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:46.353366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:46.353366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:46.353366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:46.353366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:46.353366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:46.353366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:46.353366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:46.353366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:46.353366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:46.353366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:46.354412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:46.355432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:46.355432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:46.355432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:46.355432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:46.355432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:46.355432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:46.355432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:46.355432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:46.363234 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 410/797] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:46.363788 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c2d58-9b36-4889-9a05-45cdb0e61f11 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:46.364036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c2d58-9b36-4889-9a05-45cdb0e61f11 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:46.364306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c2d58-9b36-4889-9a05-45cdb0e61f11 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:46.369693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 411/798] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:46.370585 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c2d58-9b36-4889-9a05-45cdb0e61f11 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:46.383058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:46.397478 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af53cd8-d2e0-4f35-98b1-4ffd61eabb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:46.398277 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af53cd8-d2e0-4f35-98b1-4ffd61eabb14 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:46.398659 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af53cd8-d2e0-4f35-98b1-4ffd61eabb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:46.399027 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af53cd8-d2e0-4f35-98b1-4ffd61eabb14 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:46.400636 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af53cd8-d2e0-4f35-98b1-4ffd61eabb14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:46.401043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af53cd8-d2e0-4f35-98b1-4ffd61eabb14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:46.402784 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af53cd8-d2e0-4f35-98b1-4ffd61eabb14 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:46.422265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 388/799] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:46.424552 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd333072-5bed-4de8-9ee7-dbfa713b9459 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:46.428055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd333072-5bed-4de8-9ee7-dbfa713b9459 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:46.428486 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd333072-5bed-4de8-9ee7-dbfa713b9459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:46.445808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-cd333072-5bed-4de8-9ee7-dbfa713b9459 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:46.447217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cd333072-5bed-4de8-9ee7-dbfa713b9459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:46.501909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c2d58-9b36-4889-9a05-45cdb0e61f11 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:46.529255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 389/800] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:17:46.530381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd333072-5bed-4de8-9ee7-dbfa713b9459 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:46.544205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4af9f9-e88d-45aa-b32d-1b4f5fd92439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:46.545168 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4af9f9-e88d-45aa-b32d-1b4f5fd92439 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:46.546972 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4af9f9-e88d-45aa-b32d-1b4f5fd92439 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:46.547118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4af9f9-e88d-45aa-b32d-1b4f5fd92439 None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:46.548255 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4af9f9-e88d-45aa-b32d-1b4f5fd92439 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:46.550665 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4af9f9-e88d-45aa-b32d-1b4f5fd92439 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:46.552040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4af9f9-e88d-45aa-b32d-1b4f5fd92439 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:46.568340 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 412/801] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:46.573855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4af9f9-e88d-45aa-b32d-1b4f5fd92439 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:46.608203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:46.669308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd333072-5bed-4de8-9ee7-dbfa713b9459 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:46.686870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4af9f9-e88d-45aa-b32d-1b4f5fd92439 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:46.693188 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:46.717391 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff259b4-2a64-457b-a104-f17b349790ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:46.718558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff259b4-2a64-457b-a104-f17b349790ee None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:46.719071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff259b4-2a64-457b-a104-f17b349790ee None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:46.719568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff259b4-2a64-457b-a104-f17b349790ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:46.724463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794415c3-4ccb-41c0-880c-0ea97d7b80dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:46.724931 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794415c3-4ccb-41c0-880c-0ea97d7b80dc None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:46.725255 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794415c3-4ccb-41c0-880c-0ea97d7b80dc None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:46.725532 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794415c3-4ccb-41c0-880c-0ea97d7b80dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:46.729978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff259b4-2a64-457b-a104-f17b349790ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:46.731807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff259b4-2a64-457b-a104-f17b349790ee None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:46.736517 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794415c3-4ccb-41c0-880c-0ea97d7b80dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-794415c3-4ccb-41c0-880c-0ea97d7b80dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:46.744385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:46.745695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:46.746541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:46.746541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:46.746541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:46.746541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:46.746541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:46.746541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:46.746541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:46.746541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:46.746541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:46.746541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:46.746541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:46.746541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:46.746541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 413/802] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:17:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:46.747236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794415c3-4ccb-41c0-880c-0ea97d7b80dc None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-2ff259b4-2a64-457b-a104-f17b349790ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:46.751059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:46.752127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:46.753338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:46.753338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:46.753338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:46.753338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:46.753338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:46.753338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:46.753338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:46.753338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:46.753338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 15:17:46.753338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:46.754940 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 390/803] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Jul 19 15:17:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:46.756450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15db06-de9b-49ac-b50b-e6f9108f6f68 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:46.756784 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15db06-de9b-49ac-b50b-e6f9108f6f68 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:46.757104 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15db06-de9b-49ac-b50b-e6f9108f6f68 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:46.762122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 391/804] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:46.762122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15db06-de9b-49ac-b50b-e6f9108f6f68 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:46.860266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:46.870695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794415c3-4ccb-41c0-880c-0ea97d7b80dc None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:46.875339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15db06-de9b-49ac-b50b-e6f9108f6f68 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:46.908169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbbb6b-1c5b-4466-8e4e-3d673bea24ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:46.909040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbbb6b-1c5b-4466-8e4e-3d673bea24ee None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:46.909366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbbb6b-1c5b-4466-8e4e-3d673bea24ee None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:46.909634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbbb6b-1c5b-4466-8e4e-3d673bea24ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:46.910494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbbb6b-1c5b-4466-8e4e-3d673bea24ee None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=bff6a2d4bc734904b2615608a86b097d) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:46.910810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbbb6b-1c5b-4466-8e4e-3d673bea24ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:46.912290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbbb6b-1c5b-4466-8e4e-3d673bea24ee None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:46.929612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 414/805] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:46 2022] GET /identity/v3/role_assignments?user.id=bff6a2d4bc734904b2615608a86b097d&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:46.929612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbbb6b-1c5b-4466-8e4e-3d673bea24ee None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:47.034116 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ad20d-f9c1-436c-9d01-93829d1e3419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:47.034920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ad20d-f9c1-436c-9d01-93829d1e3419 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:47.035630 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ad20d-f9c1-436c-9d01-93829d1e3419 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:47.035917 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ad20d-f9c1-436c-9d01-93829d1e3419 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:47.036981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ad20d-f9c1-436c-9d01-93829d1e3419 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:47.037317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ad20d-f9c1-436c-9d01-93829d1e3419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:47.039191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ad20d-f9c1-436c-9d01-93829d1e3419 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:47.051471 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 392/806] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:47.053517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c125a62-3c78-49fd-8791-0de8879369c4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:47.054082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c125a62-3c78-49fd-8791-0de8879369c4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:47.054363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c125a62-3c78-49fd-8791-0de8879369c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:47.059714 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbbb6b-1c5b-4466-8e4e-3d673bea24ee None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:47.073886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2c125a62-3c78-49fd-8791-0de8879369c4 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:47.075203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2c125a62-3c78-49fd-8791-0de8879369c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:47.100321 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f781db-a013-433e-a74b-5faa0bbf43b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:47.101012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f781db-a013-433e-a74b-5faa0bbf43b3 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:47.101347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f781db-a013-433e-a74b-5faa0bbf43b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:47.101627 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f781db-a013-433e-a74b-5faa0bbf43b3 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:47.102636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f781db-a013-433e-a74b-5faa0bbf43b3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:47.102947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f781db-a013-433e-a74b-5faa0bbf43b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:47.104542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f781db-a013-433e-a74b-5faa0bbf43b3 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:47.124747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:47.126558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 415/807] 10.222.0.42 () {56 vars in 1149 bytes} [Tue Jul 19 15:17:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:47.127630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f781db-a013-433e-a74b-5faa0bbf43b3 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:47.150753 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:47.181835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 393/808] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:17:47.183001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c125a62-3c78-49fd-8791-0de8879369c4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:47.268360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f781db-a013-433e-a74b-5faa0bbf43b3 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:47.306826 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d52e67-49f7-49b6-8468-a693fb6f449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:47.307638 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d52e67-49f7-49b6-8468-a693fb6f449c None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:47.307919 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d52e67-49f7-49b6-8468-a693fb6f449c None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:47.308193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d52e67-49f7-49b6-8468-a693fb6f449c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:47.315603 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c125a62-3c78-49fd-8791-0de8879369c4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:47.319649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d52e67-49f7-49b6-8468-a693fb6f449c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:47.321108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d52e67-49f7-49b6-8468-a693fb6f449c None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:47.341431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-79d52e67-49f7-49b6-8468-a693fb6f449c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:47.341431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:47.341431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:47.341431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:47.341431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:47.341431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:47.341431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:47.341431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:47.341431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:47.341431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:47.341431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:47.341431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:47.341431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:47.341431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:47.341431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:47.342568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:47.343685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:47.343685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:47.343685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:47.343685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:47.347314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 416/809] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:47.348205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d52e67-49f7-49b6-8468-a693fb6f449c None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:47.363870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc29ff-cf8c-4457-88c3-ba42d3491819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:47.364994 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc29ff-cf8c-4457-88c3-ba42d3491819 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:47.365810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc29ff-cf8c-4457-88c3-ba42d3491819 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:47.366202 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc29ff-cf8c-4457-88c3-ba42d3491819 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:47.385341 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc29ff-cf8c-4457-88c3-ba42d3491819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:47.387746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc29ff-cf8c-4457-88c3-ba42d3491819 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:47.412979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-1ddc29ff-cf8c-4457-88c3-ba42d3491819 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:47.412979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:47.412979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:47.412979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:47.412979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:47.412979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:47.412979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:47.412979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:47.412979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:47.412979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:47.412979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:47.412979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:47.412979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:47.412979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:47.412979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:47.414747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:47.415933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:47.415933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:47.415933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:47.415933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:47.417218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 394/810] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:47.418126 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc29ff-cf8c-4457-88c3-ba42d3491819 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:47.437422 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:47.442109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:47.485884 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d52e67-49f7-49b6-8468-a693fb6f449c None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:47.530995 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979c1b6-cd11-4aa6-aac3-269d9f9484b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:47.532328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979c1b6-cd11-4aa6-aac3-269d9f9484b4 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:47.532615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979c1b6-cd11-4aa6-aac3-269d9f9484b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:47.533348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979c1b6-cd11-4aa6-aac3-269d9f9484b4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:47.534753 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979c1b6-cd11-4aa6-aac3-269d9f9484b4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:47.536831 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979c1b6-cd11-4aa6-aac3-269d9f9484b4 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:47.544484 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc29ff-cf8c-4457-88c3-ba42d3491819 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-3979c1b6-cd11-4aa6-aac3-269d9f9484b4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:47.554044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:47.555210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:47.556853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 417/811] 10.222.0.42 () {56 vars in 1129 bytes} [Tue Jul 19 15:17:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:47.558214 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979c1b6-cd11-4aa6-aac3-269d9f9484b4 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:47.584568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e31937-0b2b-4484-8c81-45f6cc356886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:47.585435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e31937-0b2b-4484-8c81-45f6cc356886 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:47.585815 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e31937-0b2b-4484-8c81-45f6cc356886 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:47.586210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e31937-0b2b-4484-8c81-45f6cc356886 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:47.587096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e31937-0b2b-4484-8c81-45f6cc356886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:47.587499 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e31937-0b2b-4484-8c81-45f6cc356886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:47.588990 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e31937-0b2b-4484-8c81-45f6cc356886 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:47.607755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 395/812] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:47.608791 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e31937-0b2b-4484-8c81-45f6cc356886 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:47.699099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979c1b6-cd11-4aa6-aac3-269d9f9484b4 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:47.703787 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:47.734203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:47.737582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e31937-0b2b-4484-8c81-45f6cc356886 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:47.740923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b6cc9-702a-4749-863b-f05c3509de82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:47.741633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b6cc9-702a-4749-863b-f05c3509de82 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:47.741915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b6cc9-702a-4749-863b-f05c3509de82 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:47.742195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b6cc9-702a-4749-863b-f05c3509de82 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:47.742924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b6cc9-702a-4749-863b-f05c3509de82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:47.743233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b6cc9-702a-4749-863b-f05c3509de82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:47.744610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b6cc9-702a-4749-863b-f05c3509de82 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:47.756912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 418/813] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:47.757830 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b6cc9-702a-4749-863b-f05c3509de82 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:47.778678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f6326-439b-4f0a-8746-7dd5ca7e4238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:47.779760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f6326-439b-4f0a-8746-7dd5ca7e4238 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:47.780358 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f6326-439b-4f0a-8746-7dd5ca7e4238 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:47.780733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f6326-439b-4f0a-8746-7dd5ca7e4238 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:47.781554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f6326-439b-4f0a-8746-7dd5ca7e4238 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:47.781865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f6326-439b-4f0a-8746-7dd5ca7e4238 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:47.784113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f6326-439b-4f0a-8746-7dd5ca7e4238 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:47.789169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 396/814] 10.222.0.42 () {56 vars in 1124 bytes} [Tue Jul 19 15:17:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:47.790490 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee11b5-56cd-4763-a638-2adbdd9daff6 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:47.790776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee11b5-56cd-4763-a638-2adbdd9daff6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:47.791052 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee11b5-56cd-4763-a638-2adbdd9daff6 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:47.796227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 397/815] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:47.796227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee11b5-56cd-4763-a638-2adbdd9daff6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:47.868509 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b6cc9-702a-4749-863b-f05c3509de82 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:47.904304 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc8c66-2f6b-4088-b0a4-1dc18f06fe2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:47.905100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc8c66-2f6b-4088-b0a4-1dc18f06fe2a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:47.905407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc8c66-2f6b-4088-b0a4-1dc18f06fe2a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:47.905745 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc8c66-2f6b-4088-b0a4-1dc18f06fe2a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:47.906844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee11b5-56cd-4763-a638-2adbdd9daff6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:47.912962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc8c66-2f6b-4088-b0a4-1dc18f06fe2a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:47.914433 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc8c66-2f6b-4088-b0a4-1dc18f06fe2a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-00cc8c66-2f6b-4088-b0a4-1dc18f06fe2a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:47.928896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:47.929879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:47.930840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:47.930840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:47.930840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:47.930840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:47.930840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:47.930840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:47.930840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:47.930840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:47.930840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 15:17:47.930840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:47.930840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 419/816] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:17:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:47.930840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc8c66-2f6b-4088-b0a4-1dc18f06fe2a None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:47.950028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dac664-8ed3-4c7a-a001-753a6b940995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:47.951054 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dac664-8ed3-4c7a-a001-753a6b940995 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:47.951493 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dac664-8ed3-4c7a-a001-753a6b940995 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:47.951770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dac664-8ed3-4c7a-a001-753a6b940995 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:47.959396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dac664-8ed3-4c7a-a001-753a6b940995 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:47.960832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dac664-8ed3-4c7a-a001-753a6b940995 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:47.975958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-22dac664-8ed3-4c7a-a001-753a6b940995 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:47.975958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:47.975958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:47.975958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:47.975958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:47.975958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:47.975958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:47.975958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:47.975958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:47.975958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:47.975958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:47.975958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:47.975958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:47.975958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:47.975958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:47.976832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:47.978072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:47.978072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:47.978072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:47.978072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:47.978072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:47.978072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:47.978072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:47.978072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:47.978072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 398/817] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:47.979071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f016b5d-278f-4cbb-85ce-4f4ac7448991 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:47.979071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f016b5d-278f-4cbb-85ce-4f4ac7448991 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:47.979071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f016b5d-278f-4cbb-85ce-4f4ac7448991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:47.983710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:47.989308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:47.997626 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9f016b5d-278f-4cbb-85ce-4f4ac7448991 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:47.998850 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9f016b5d-278f-4cbb-85ce-4f4ac7448991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:48.042984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc8c66-2f6b-4088-b0a4-1dc18f06fe2a None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:48.072498 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 399/818] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:17:48.073344 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f016b5d-278f-4cbb-85ce-4f4ac7448991 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:48.089090 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416bdf3-686e-4185-8e82-ad05f4935990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:48.090283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416bdf3-686e-4185-8e82-ad05f4935990 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:48.090793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416bdf3-686e-4185-8e82-ad05f4935990 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:48.091475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416bdf3-686e-4185-8e82-ad05f4935990 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:48.092769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416bdf3-686e-4185-8e82-ad05f4935990 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:48.093454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416bdf3-686e-4185-8e82-ad05f4935990 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:48.096078 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416bdf3-686e-4185-8e82-ad05f4935990 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:48.104084 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 420/819] 10.222.0.42 () {56 vars in 1124 bytes} [Tue Jul 19 15:17:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:48.105549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416bdf3-686e-4185-8e82-ad05f4935990 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:48.183382 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f016b5d-278f-4cbb-85ce-4f4ac7448991 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:48.217097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:48.223385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416bdf3-686e-4185-8e82-ad05f4935990 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:48.223698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ec558-4ace-4378-86f3-3493de676d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:48.224386 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ec558-4ace-4378-86f3-3493de676d66 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:48.224668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ec558-4ace-4378-86f3-3493de676d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:48.224945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ec558-4ace-4378-86f3-3493de676d66 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:48.225840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ec558-4ace-4378-86f3-3493de676d66 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:48.226139 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ec558-4ace-4378-86f3-3493de676d66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:48.227564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ec558-4ace-4378-86f3-3493de676d66 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:48.242432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:48.246796 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 400/820] 10.222.0.42 () {56 vars in 1135 bytes} [Tue Jul 19 15:17:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:48.248374 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ec558-4ace-4378-86f3-3493de676d66 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:48.275658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd941c-de5d-43dd-9df9-e30c4d745654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:48.276336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd941c-de5d-43dd-9df9-e30c4d745654 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:48.276632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd941c-de5d-43dd-9df9-e30c4d745654 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:48.276899 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd941c-de5d-43dd-9df9-e30c4d745654 None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:48.277863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd941c-de5d-43dd-9df9-e30c4d745654 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:48.278179 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd941c-de5d-43dd-9df9-e30c4d745654 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:48.279627 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd941c-de5d-43dd-9df9-e30c4d745654 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:48.297161 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 421/821] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:48.297161 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd941c-de5d-43dd-9df9-e30c4d745654 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:48.363586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ec558-4ace-4378-86f3-3493de676d66 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:48.399448 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade776f-d835-4824-bf4a-4c7752787e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:48.400509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade776f-d835-4824-bf4a-4c7752787e6e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:48.400793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade776f-d835-4824-bf4a-4c7752787e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:48.401069 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade776f-d835-4824-bf4a-4c7752787e6e None admin] PATH_INFO: `/v3/services/network` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:48.401757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade776f-d835-4824-bf4a-4c7752787e6e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:48.403114 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade776f-d835-4824-bf4a-4c7752787e6e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:48.405113 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd941c-de5d-43dd-9df9-e30c4d745654 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-dade776f-d835-4824-bf4a-4c7752787e6e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:48.415009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:48.416215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:48.417081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:48.417081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:48.417081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:48.417081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:48.417081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:48.417081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:48.417081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:48.417081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:48.417081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:48.417081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:48.417081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:17:48.417081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:48.417081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 15:17:48.417081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:48.417081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 401/822] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:48 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:48.418078 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade776f-d835-4824-bf4a-4c7752787e6e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:48.446774 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90e3de-a9fe-454d-b186-9f83dd39bb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:48.447648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90e3de-a9fe-454d-b186-9f83dd39bb18 None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:48.447934 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90e3de-a9fe-454d-b186-9f83dd39bb18 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:48.448207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90e3de-a9fe-454d-b186-9f83dd39bb18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:48.448940 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90e3de-a9fe-454d-b186-9f83dd39bb18 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:48.450328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90e3de-a9fe-454d-b186-9f83dd39bb18 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:48.481520 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 422/823] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Jul 19 15:17:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:17:48.481520 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90e3de-a9fe-454d-b186-9f83dd39bb18 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:48.486709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:48.535369 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:48.550120 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade776f-d835-4824-bf4a-4c7752787e6e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:48.601400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcbb53-31c2-4b26-b139-f5b38ba18855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:48.602381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcbb53-31c2-4b26-b139-f5b38ba18855 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:48.602760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcbb53-31c2-4b26-b139-f5b38ba18855 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:48.603130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcbb53-31c2-4b26-b139-f5b38ba18855 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:48.614891 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcbb53-31c2-4b26-b139-f5b38ba18855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:48.616516 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcbb53-31c2-4b26-b139-f5b38ba18855 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:48.636586 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90e3de-a9fe-454d-b186-9f83dd39bb18 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:48.637006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-64fcbb53-31c2-4b26-b139-f5b38ba18855 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:48.637006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:48.637006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:48.637006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:48.637006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:48.637006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:48.637006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:48.637006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:48.637006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:48.637006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:48.637006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:48.637006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:48.637006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:48.637006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:48.637006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:48.638066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:48.638964 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:48.638964 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:48.638964 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:48.638964 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:48.639458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 402/824] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:48.640678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcbb53-31c2-4b26-b139-f5b38ba18855 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:48.691136 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd2f11-3b85-44d5-a560-2525d97965c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:48.692110 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd2f11-3b85-44d5-a560-2525d97965c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:48.692411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd2f11-3b85-44d5-a560-2525d97965c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:48.692695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd2f11-3b85-44d5-a560-2525d97965c2 None admin] PATH_INFO: `/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/60f89f6042664f91b2829d516ed7e844/roles/c8578ede4e7041ecaeba79c255cc650b` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:48.712606 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd2f11-3b85-44d5-a560-2525d97965c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5f51e47b8e34bf5ac822d8c93853e30, user_id=60f89f6042664f91b2829d516ed7e844, role_id=c8578ede4e7041ecaeba79c255cc650b)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:48.713455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd2f11-3b85-44d5-a560-2525d97965c2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:48.731979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 423/825] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:17:48 2022] PUT /identity/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/60f89f6042664f91b2829d516ed7e844/roles/c8578ede4e7041ecaeba79c255cc650b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:17:48.732531 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd2f11-3b85-44d5-a560-2525d97965c2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:48.785612 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcbb53-31c2-4b26-b139-f5b38ba18855 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:48.787678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:48.826007 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d1f91-d37d-41dd-a40d-924351fb8e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:48.826959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d1f91-d37d-41dd-a40d-924351fb8e06 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:48.827274 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d1f91-d37d-41dd-a40d-924351fb8e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:48.827632 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d1f91-d37d-41dd-a40d-924351fb8e06 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:48.828461 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d1f91-d37d-41dd-a40d-924351fb8e06 None admin] RBAC: Adding query filter params (name=network) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:48.828881 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d1f91-d37d-41dd-a40d-924351fb8e06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:48.831063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d1f91-d37d-41dd-a40d-924351fb8e06 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:48.836532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 403/826] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Jul 19 15:17:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:48.845618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d1f91-d37d-41dd-a40d-924351fb8e06 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:48.868970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:48.896674 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd2f11-3b85-44d5-a560-2525d97965c2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:48.970244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d0579-db14-4cc5-acf8-c1abf18ad7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:48.971262 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d0579-db14-4cc5-acf8-c1abf18ad7ea None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:48.971696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d0579-db14-4cc5-acf8-c1abf18ad7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:48.972115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d0579-db14-4cc5-acf8-c1abf18ad7ea None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:48.972992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d0579-db14-4cc5-acf8-c1abf18ad7ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:48.974012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d0579-db14-4cc5-acf8-c1abf18ad7ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:48.976023 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d0579-db14-4cc5-acf8-c1abf18ad7ea None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:48.992989 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 424/827] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:49.001913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d0579-db14-4cc5-acf8-c1abf18ad7ea None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:49.013370 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d1f91-d37d-41dd-a40d-924351fb8e06 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:49.060578 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717eb02b-0e60-4782-b905-71f2ab469769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:49.061442 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eb02b-0e60-4782-b905-71f2ab469769 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:49.061827 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eb02b-0e60-4782-b905-71f2ab469769 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:49.062235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eb02b-0e60-4782-b905-71f2ab469769 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:49.063296 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717eb02b-0e60-4782-b905-71f2ab469769 None admin] RBAC: Adding query filter params (type=network) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:49.063613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717eb02b-0e60-4782-b905-71f2ab469769 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:49.066601 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717eb02b-0e60-4782-b905-71f2ab469769 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:49.073709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 404/828] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Jul 19 15:17:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:49.082201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717eb02b-0e60-4782-b905-71f2ab469769 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:49.121800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:49.136058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d0579-db14-4cc5-acf8-c1abf18ad7ea None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:49.173966 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a74a76-e86a-4ba9-9634-610b7ba52da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:49.175170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a74a76-e86a-4ba9-9634-610b7ba52da7 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:49.175669 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a74a76-e86a-4ba9-9634-610b7ba52da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:49.176163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a74a76-e86a-4ba9-9634-610b7ba52da7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:49.183806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a74a76-e86a-4ba9-9634-610b7ba52da7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:49.185530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a74a76-e86a-4ba9-9634-610b7ba52da7 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:49.188666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:49.201539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717eb02b-0e60-4782-b905-71f2ab469769 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:49.201731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-e2a74a76-e86a-4ba9-9634-610b7ba52da7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:49.201731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:49.201731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:49.201731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:49.201731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:49.201731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:49.201731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:49.201731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:49.201731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:49.201731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:49.201731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:49.201731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:49.201731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:49.201731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:49.201731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:49.202550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:49.204250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:49.204250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:49.204250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:49.204250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:49.204250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:49.204250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:49.204250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:49.204250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:49.204250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 425/829] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:49.205497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32202f-f9d1-475b-a278-805dd8a28686 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:49.205497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32202f-f9d1-475b-a278-805dd8a28686 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:49.205776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32202f-f9d1-475b-a278-805dd8a28686 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:49.207224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 426/830] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:49.210061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32202f-f9d1-475b-a278-805dd8a28686 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:49.256253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ce110-3a11-498c-9a35-fafce3b956ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:49.257302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ce110-3a11-498c-9a35-fafce3b956ee None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:49.258100 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ce110-3a11-498c-9a35-fafce3b956ee None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:49.259669 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ce110-3a11-498c-9a35-fafce3b956ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:49.260645 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ce110-3a11-498c-9a35-fafce3b956ee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:49.262327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ce110-3a11-498c-9a35-fafce3b956ee None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:49.297163 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 405/831] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Jul 19 15:17:49 2022] POST /identity/v3/endpoints => generated 327 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:17:49.299014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836dba9-cfe1-4188-b534-fd9ff39da9e2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:49.299311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836dba9-cfe1-4188-b534-fd9ff39da9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:49.299595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836dba9-cfe1-4188-b534-fd9ff39da9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:49.341656 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b836dba9-cfe1-4188-b534-fd9ff39da9e2 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:49.343963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b836dba9-cfe1-4188-b534-fd9ff39da9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:49.393935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32202f-f9d1-475b-a278-805dd8a28686 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:49.468202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f051451-a60a-47e4-b6f4-ddcd359b2595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:49.468885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f051451-a60a-47e4-b6f4-ddcd359b2595 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:49.469187 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f051451-a60a-47e4-b6f4-ddcd359b2595 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:49.469471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f051451-a60a-47e4-b6f4-ddcd359b2595 None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:49.470827 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f051451-a60a-47e4-b6f4-ddcd359b2595 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:49.471137 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f051451-a60a-47e4-b6f4-ddcd359b2595 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:49.472625 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f051451-a60a-47e4-b6f4-ddcd359b2595 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:49.492433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 406/832] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:17:49.498588 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 427/833] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:49.504637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836dba9-cfe1-4188-b534-fd9ff39da9e2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:49.516326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f051451-a60a-47e4-b6f4-ddcd359b2595 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:49.533619 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:49.592431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:49.618957 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836dba9-cfe1-4188-b534-fd9ff39da9e2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:49.645728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f051451-a60a-47e4-b6f4-ddcd359b2595 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:49.655896 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d6f79-3928-4186-8842-142ba29da623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:49.656795 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d6f79-3928-4186-8842-142ba29da623 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:49.657080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d6f79-3928-4186-8842-142ba29da623 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:49.657388 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d6f79-3928-4186-8842-142ba29da623 None admin] PATH_INFO: `/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/96faaac55bc5408685c68a6177d91971/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:49.659989 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d6f79-3928-4186-8842-142ba29da623 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5f51e47b8e34bf5ac822d8c93853e30, user_id=96faaac55bc5408685c68a6177d91971, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:49.662153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d6f79-3928-4186-8842-142ba29da623 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:49.678731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 407/834] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:17:49 2022] PUT /identity/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/96faaac55bc5408685c68a6177d91971/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:17:49.699298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f05a9-8e2d-471a-a42d-152467d8ff19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:49.700535 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f05a9-8e2d-471a-a42d-152467d8ff19 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:49.701188 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f05a9-8e2d-471a-a42d-152467d8ff19 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:49.702271 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f05a9-8e2d-471a-a42d-152467d8ff19 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:49.703267 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f05a9-8e2d-471a-a42d-152467d8ff19 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:49.705617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f05a9-8e2d-471a-a42d-152467d8ff19 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-037f05a9-8e2d-471a-a42d-152467d8ff19 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:49.722942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:49.723962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:49.724837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:49.724837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:49.724837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:49.724837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:49.724837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:49.724837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:49.724837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:49.724837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:49.724837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:49.724837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:49.724837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:49.724837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:17:49.724837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:49.724837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 15:17:49.724837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:49.728075 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 428/835] 10.222.0.42 () {56 vars in 1132 bytes} [Tue Jul 19 15:17:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:49.734413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d6f79-3928-4186-8842-142ba29da623 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:49.840480 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:49.865833 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d6f79-3928-4186-8842-142ba29da623 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:49.903280 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33bea0-51b8-4482-b2df-36688178c33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:49.903952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33bea0-51b8-4482-b2df-36688178c33c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:49.904237 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33bea0-51b8-4482-b2df-36688178c33c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:49.904514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33bea0-51b8-4482-b2df-36688178c33c None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:49.905265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33bea0-51b8-4482-b2df-36688178c33c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:49.905578 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33bea0-51b8-4482-b2df-36688178c33c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:49.906950 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33bea0-51b8-4482-b2df-36688178c33c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:49.913102 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 408/836] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Jul 19 15:17:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:49.918853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f05a9-8e2d-471a-a42d-152467d8ff19 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:50.026575 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f05a9-8e2d-471a-a42d-152467d8ff19 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:50.047464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:50.062292 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541fb19-3421-4b0d-8c61-950797f4416d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:50.063029 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541fb19-3421-4b0d-8c61-950797f4416d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:50.063318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541fb19-3421-4b0d-8c61-950797f4416d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:50.063594 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541fb19-3421-4b0d-8c61-950797f4416d None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:50.065046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541fb19-3421-4b0d-8c61-950797f4416d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:50.065405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541fb19-3421-4b0d-8c61-950797f4416d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:50.066797 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541fb19-3421-4b0d-8c61-950797f4416d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:50.071958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 429/837] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Jul 19 15:17:49 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:50.870096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb440187-4694-4226-b3a4-4369f0333a27 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:50.872398 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb440187-4694-4226-b3a4-4369f0333a27 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:50.872398 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb440187-4694-4226-b3a4-4369f0333a27 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:50.872656 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 409/838] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:50.878389 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de54c46-8f08-48ac-8c14-7b3b60f3dfbe None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:50.878913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de54c46-8f08-48ac-8c14-7b3b60f3dfbe None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:50.878913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de54c46-8f08-48ac-8c14-7b3b60f3dfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:50.896507 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-9de54c46-8f08-48ac-8c14-7b3b60f3dfbe None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:50.897728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9de54c46-8f08-48ac-8c14-7b3b60f3dfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:50.972509 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 430/839] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:17:50.994432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb440187-4694-4226-b3a4-4369f0333a27 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:51.099038 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb440187-4694-4226-b3a4-4369f0333a27 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:51.114164 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cce73-718a-4683-b04e-66b60431107a None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:51.114164 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cce73-718a-4683-b04e-66b60431107a None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:51.114164 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cce73-718a-4683-b04e-66b60431107a None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:51.114164 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 431/840] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:17:51.121633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01541a8b-125f-4d9f-8823-1f63f544480b None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:51.121633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01541a8b-125f-4d9f-8823-1f63f544480b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:51.121633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01541a8b-125f-4d9f-8823-1f63f544480b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:51.136865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c302ae-4f62-43d2-80fc-47c1b6a866a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:51.137962 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c302ae-4f62-43d2-80fc-47c1b6a866a0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:51.138118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-01541a8b-125f-4d9f-8823-1f63f544480b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:51.138580 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c302ae-4f62-43d2-80fc-47c1b6a866a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:51.139034 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c302ae-4f62-43d2-80fc-47c1b6a866a0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:51.139581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-01541a8b-125f-4d9f-8823-1f63f544480b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:51.140044 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c302ae-4f62-43d2-80fc-47c1b6a866a0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:51.141757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c302ae-4f62-43d2-80fc-47c1b6a866a0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-89c302ae-4f62-43d2-80fc-47c1b6a866a0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:51.154216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:51.155507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:51.156565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:51.156565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:51.156565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:51.156565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:51.156565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:51.156565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:51.156565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:51.156565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:51.156565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:51.156565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:51.156565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:17:51.156565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:51.156565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 15:17:51.156565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:51.159627 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 410/841] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:17:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:51.166084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c302ae-4f62-43d2-80fc-47c1b6a866a0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:51.169954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:51.224988 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 432/842] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:17:51.248836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01541a8b-125f-4d9f-8823-1f63f544480b None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:51.293089 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c302ae-4f62-43d2-80fc-47c1b6a866a0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:51.312385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:51.331575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956e839-0e7c-4ad0-85fb-76e8a92bce46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:51.332368 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956e839-0e7c-4ad0-85fb-76e8a92bce46 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:51.332655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956e839-0e7c-4ad0-85fb-76e8a92bce46 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:51.332897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956e839-0e7c-4ad0-85fb-76e8a92bce46 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:51.333635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956e839-0e7c-4ad0-85fb-76e8a92bce46 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:51.333944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956e839-0e7c-4ad0-85fb-76e8a92bce46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:51.335372 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956e839-0e7c-4ad0-85fb-76e8a92bce46 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:51.340688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 411/843] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:17:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:51.350123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956e839-0e7c-4ad0-85fb-76e8a92bce46 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:51.379626 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01541a8b-125f-4d9f-8823-1f63f544480b None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:51.415571 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadb5e5-0468-4b69-94d6-366a0abb8ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:51.416367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadb5e5-0468-4b69-94d6-366a0abb8ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:51.416646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadb5e5-0468-4b69-94d6-366a0abb8ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:51.416924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadb5e5-0468-4b69-94d6-366a0abb8ce2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:51.427609 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cadb5e5-0468-4b69-94d6-366a0abb8ce2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:51.428976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-9cadb5e5-0468-4b69-94d6-366a0abb8ce2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:51.435928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:51.436898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:51.437773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:51.437773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:51.437773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:51.437773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:51.437773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:51.437773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:51.437773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:51.437773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:51.437773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:51.437773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:51.437773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:17:51.437773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:51.442108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 433/844] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:17:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:51.443390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadb5e5-0468-4b69-94d6-366a0abb8ce2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:51.475402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956e839-0e7c-4ad0-85fb-76e8a92bce46 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:51.513399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb57d0-70d1-4e5a-a41c-08b7d87ec778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:51.514275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb57d0-70d1-4e5a-a41c-08b7d87ec778 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:51.514558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb57d0-70d1-4e5a-a41c-08b7d87ec778 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:51.514833 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb57d0-70d1-4e5a-a41c-08b7d87ec778 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:51.515644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb57d0-70d1-4e5a-a41c-08b7d87ec778 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:51.515955 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb57d0-70d1-4e5a-a41c-08b7d87ec778 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:51.518072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb57d0-70d1-4e5a-a41c-08b7d87ec778 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:51.529529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 412/845] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:17:51 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:51.563610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadb5e5-0468-4b69-94d6-366a0abb8ce2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:51.567241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:51.607969 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402c407-6975-4f96-810e-17f82b2a44ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:51.609075 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402c407-6975-4f96-810e-17f82b2a44ef None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:51.609615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402c407-6975-4f96-810e-17f82b2a44ef None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:51.610089 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402c407-6975-4f96-810e-17f82b2a44ef None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:51.611371 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402c407-6975-4f96-810e-17f82b2a44ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:51.611936 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402c407-6975-4f96-810e-17f82b2a44ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:51.613611 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402c407-6975-4f96-810e-17f82b2a44ef None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:51.628976 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 434/846] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:17:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:51.630829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb57d0-70d1-4e5a-a41c-08b7d87ec778 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:51.742021 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb57d0-70d1-4e5a-a41c-08b7d87ec778 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:51.777542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:51.778591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7db413-cacb-4916-aa27-16a0e5be13c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:51.779494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7db413-cacb-4916-aa27-16a0e5be13c9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:51.780122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7db413-cacb-4916-aa27-16a0e5be13c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:51.780505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7db413-cacb-4916-aa27-16a0e5be13c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:51.792708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7db413-cacb-4916-aa27-16a0e5be13c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:51.794346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7db413-cacb-4916-aa27-16a0e5be13c9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:51.812083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-2d7db413-cacb-4916-aa27-16a0e5be13c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:51.812083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:51.812083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:51.812083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:51.812083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:51.812083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:51.812083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:51.812083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:51.812083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:51.812083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:51.812083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:51.812083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:51.812083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:51.812083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:51.812083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:51.813315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:51.814959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:51.814959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:51.814959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:51.814959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:51.821169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 413/847] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:51.822576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402c407-6975-4f96-810e-17f82b2a44ef None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:51.849986 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861d809-1ca2-46f0-8957-27a492ea64bb None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:51.850278 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861d809-1ca2-46f0-8957-27a492ea64bb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:51.850564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861d809-1ca2-46f0-8957-27a492ea64bb None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:51.852225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 414/848] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:17:51.857103 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e4bf0-420f-40d7-9ca8-919d4fbf853d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:51.857428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e4bf0-420f-40d7-9ca8-919d4fbf853d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:51.857715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e4bf0-420f-40d7-9ca8-919d4fbf853d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:51.878693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d79e4bf0-420f-40d7-9ca8-919d4fbf853d None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:51.880915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d79e4bf0-420f-40d7-9ca8-919d4fbf853d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:51.941018 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402c407-6975-4f96-810e-17f82b2a44ef None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:51.958334 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 415/849] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:17:51.982645 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e4bf0-420f-40d7-9ca8-919d4fbf853d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:51.991798 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773527b-c3c5-4de4-bd66-07a5200251f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:51.992778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773527b-c3c5-4de4-bd66-07a5200251f7 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:51.993755 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773527b-c3c5-4de4-bd66-07a5200251f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:51.994263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773527b-c3c5-4de4-bd66-07a5200251f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:51.995651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773527b-c3c5-4de4-bd66-07a5200251f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:51.996443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773527b-c3c5-4de4-bd66-07a5200251f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:51.998341 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773527b-c3c5-4de4-bd66-07a5200251f7 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:52.017911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 435/850] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:52.018913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773527b-c3c5-4de4-bd66-07a5200251f7 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:52.046092 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:52.100343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e4bf0-420f-40d7-9ca8-919d4fbf853d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:52.117385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:52.122336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773527b-c3c5-4de4-bd66-07a5200251f7 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:52.141367 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f84ea-2087-4e71-8d88-92ab10de16bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:52.143055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f84ea-2087-4e71-8d88-92ab10de16bd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:52.143563 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f84ea-2087-4e71-8d88-92ab10de16bd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:52.144570 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f84ea-2087-4e71-8d88-92ab10de16bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:52.158828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef981cf-903d-4bf6-9072-d3530e4648c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:52.159583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f84ea-2087-4e71-8d88-92ab10de16bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:52.159723 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef981cf-903d-4bf6-9072-d3530e4648c2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:52.159896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef981cf-903d-4bf6-9072-d3530e4648c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:52.160162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef981cf-903d-4bf6-9072-d3530e4648c2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:52.167851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef981cf-903d-4bf6-9072-d3530e4648c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:52.169321 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef981cf-903d-4bf6-9072-d3530e4648c2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-af9f84ea-2087-4e71-8d88-92ab10de16bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:52.169724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:52.170840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:52.171689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:52.171689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:52.171689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:52.171689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:52.171689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:52.171689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:52.171689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:52.171689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:52.171689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:52.171689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:52.171689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:17:52.171689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:52.177281 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 416/851] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:17:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:52.179106 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f84ea-2087-4e71-8d88-92ab10de16bd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-5ef981cf-903d-4bf6-9072-d3530e4648c2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:52.192126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:52.193120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:52.194218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:52.194218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:52.194218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:52.194218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:52.194218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:52.194218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:52.194218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:52.194218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:52.194218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 15:17:52.194218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:52.194218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 436/852] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Jul 19 15:17:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:52.199145 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593dce6-b47a-4579-bc73-de0c3257f655 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:52.199426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593dce6-b47a-4579-bc73-de0c3257f655 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:52.199860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593dce6-b47a-4579-bc73-de0c3257f655 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:52.201775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 437/853] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:52.202373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593dce6-b47a-4579-bc73-de0c3257f655 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:52.297383 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:52.300156 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f84ea-2087-4e71-8d88-92ab10de16bd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:52.316815 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593dce6-b47a-4579-bc73-de0c3257f655 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:52.336956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56e52e-469f-4600-b6f1-1b20eb18669c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:52.338007 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56e52e-469f-4600-b6f1-1b20eb18669c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:52.338502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56e52e-469f-4600-b6f1-1b20eb18669c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:52.338842 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56e52e-469f-4600-b6f1-1b20eb18669c None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:52.339593 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56e52e-469f-4600-b6f1-1b20eb18669c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:52.339900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56e52e-469f-4600-b6f1-1b20eb18669c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:52.341279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56e52e-469f-4600-b6f1-1b20eb18669c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:52.349429 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 417/854] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:17:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:52.350696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630cb53-d6fc-401b-b079-9805865a56c2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:52.351003 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630cb53-d6fc-401b-b079-9805865a56c2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:52.351280 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630cb53-d6fc-401b-b079-9805865a56c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:52.358530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c8eed-3a81-45e0-86db-448df9181bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:52.359211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c8eed-3a81-45e0-86db-448df9181bbb None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:52.359499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c8eed-3a81-45e0-86db-448df9181bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:52.359780 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c8eed-3a81-45e0-86db-448df9181bbb None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:52.360640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c8eed-3a81-45e0-86db-448df9181bbb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:52.360988 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c8eed-3a81-45e0-86db-448df9181bbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:52.362471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c8eed-3a81-45e0-86db-448df9181bbb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:52.368359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e630cb53-d6fc-401b-b079-9805865a56c2 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:52.369572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e630cb53-d6fc-401b-b079-9805865a56c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:52.382091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 438/855] 10.222.0.42 () {56 vars in 1149 bytes} [Tue Jul 19 15:17:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:52.382773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c8eed-3a81-45e0-86db-448df9181bbb None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:52.405580 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:52.449500 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 418/856] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:52 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:17:52.450336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630cb53-d6fc-401b-b079-9805865a56c2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:52.493115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c8eed-3a81-45e0-86db-448df9181bbb None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:52.535429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efae772-6f7d-4916-9142-96b3a3aa91c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:52.536224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efae772-6f7d-4916-9142-96b3a3aa91c3 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:52.536506 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efae772-6f7d-4916-9142-96b3a3aa91c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:52.536782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efae772-6f7d-4916-9142-96b3a3aa91c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:52.546858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efae772-6f7d-4916-9142-96b3a3aa91c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:52.548923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efae772-6f7d-4916-9142-96b3a3aa91c3 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:52.551473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:52.569390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630cb53-d6fc-401b-b079-9805865a56c2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:52.574704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-0efae772-6f7d-4916-9142-96b3a3aa91c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:52.574704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:52.574704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:52.574704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:52.574704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:52.574704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:52.574704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:52.574704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:52.574704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:52.574704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:52.574704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:52.574704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:52.574704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:52.574704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:52.574704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:52.575605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:52.576514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:52.576514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:52.576514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:52.576514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:52.576514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 439/857] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:52.576809 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efae772-6f7d-4916-9142-96b3a3aa91c3 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:52.610411 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca36db-ccac-4932-af1e-0451ab76d98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:52.612055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca36db-ccac-4932-af1e-0451ab76d98c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:52.612055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca36db-ccac-4932-af1e-0451ab76d98c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:52.612251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca36db-ccac-4932-af1e-0451ab76d98c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:52.656227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:52.697269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efae772-6f7d-4916-9142-96b3a3aa91c3 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:52.698445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca36db-ccac-4932-af1e-0451ab76d98c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:52.700119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca36db-ccac-4932-af1e-0451ab76d98c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:52.733862 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5bdc5-9051-48ce-8b3f-283ebab186a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:52.734538 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5bdc5-9051-48ce-8b3f-283ebab186a1 None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:52.734827 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5bdc5-9051-48ce-8b3f-283ebab186a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:52.735117 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5bdc5-9051-48ce-8b3f-283ebab186a1 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:52.736467 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5bdc5-9051-48ce-8b3f-283ebab186a1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:52.737987 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5bdc5-9051-48ce-8b3f-283ebab186a1 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:52.756528 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 440/858] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Jul 19 15:17:52 2022] POST /identity/v3/services => generated 269 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:17:52.757632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5bdc5-9051-48ce-8b3f-283ebab186a1 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:52.835487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 419/859] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:52 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:17:52.838011 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca36db-ccac-4932-af1e-0451ab76d98c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:52.855999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:52.918886 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5bdc5-9051-48ce-8b3f-283ebab186a1 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:52.939283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:52.967328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9239ba3-f31d-432a-a0e7-956ddcf6a6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:52.968000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9239ba3-f31d-432a-a0e7-956ddcf6a6ad None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:52.968282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9239ba3-f31d-432a-a0e7-956ddcf6a6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:52.968558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9239ba3-f31d-432a-a0e7-956ddcf6a6ad None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:52.969343 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9239ba3-f31d-432a-a0e7-956ddcf6a6ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:52.969652 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9239ba3-f31d-432a-a0e7-956ddcf6a6ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:52.971723 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9239ba3-f31d-432a-a0e7-956ddcf6a6ad None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:52.984683 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 441/860] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:52.985402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca36db-ccac-4932-af1e-0451ab76d98c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:52.990157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9239ba3-f31d-432a-a0e7-956ddcf6a6ad None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:53.028889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7af4d-bab6-4228-a838-ff14bd0ff135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:53.029799 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7af4d-bab6-4228-a838-ff14bd0ff135 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:53.030081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7af4d-bab6-4228-a838-ff14bd0ff135 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:53.030350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7af4d-bab6-4228-a838-ff14bd0ff135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:53.041351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7af4d-bab6-4228-a838-ff14bd0ff135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:53.042886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7af4d-bab6-4228-a838-ff14bd0ff135 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:53.061085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f5c7af4d-bab6-4228-a838-ff14bd0ff135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:53.061085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:53.061085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:53.061085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:53.061085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:53.061085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:53.061085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:53.061085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:53.061085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:53.061085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:53.061085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:53.061085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:53.061085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:53.061085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:53.061085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:53.062037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:53.062944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:53.062944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:53.062944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:53.062944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:53.062944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 420/861] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:53.069677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7af4d-bab6-4228-a838-ff14bd0ff135 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:53.107480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9239ba3-f31d-432a-a0e7-956ddcf6a6ad None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:53.123677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:53.147741 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d316b26-eff5-406b-b2eb-f6bb1be86737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:53.148542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d316b26-eff5-406b-b2eb-f6bb1be86737 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:53.148840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d316b26-eff5-406b-b2eb-f6bb1be86737 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:53.149112 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d316b26-eff5-406b-b2eb-f6bb1be86737 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:53.158226 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d316b26-eff5-406b-b2eb-f6bb1be86737 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:53.160351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d316b26-eff5-406b-b2eb-f6bb1be86737 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-2d316b26-eff5-406b-b2eb-f6bb1be86737 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:53.177280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:53.178237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:53.179200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:53.179200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:53.179200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:53.179200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:53.179200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:53.179200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:53.179200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:53.179200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:53.179200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 15:17:53.179200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:53.179200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 442/862] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:17:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:53.185360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d316b26-eff5-406b-b2eb-f6bb1be86737 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:53.189303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7af4d-bab6-4228-a838-ff14bd0ff135 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:53.208052 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:53.228166 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033076d-86fb-4769-909f-54beed969e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:53.229215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033076d-86fb-4769-909f-54beed969e5e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:53.229844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033076d-86fb-4769-909f-54beed969e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:53.230319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033076d-86fb-4769-909f-54beed969e5e None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:53.231336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033076d-86fb-4769-909f-54beed969e5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:53.231849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033076d-86fb-4769-909f-54beed969e5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:53.233574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033076d-86fb-4769-909f-54beed969e5e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:53.254238 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 421/863] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:53.255492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033076d-86fb-4769-909f-54beed969e5e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:53.309608 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d316b26-eff5-406b-b2eb-f6bb1be86737 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:53.348739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c61be-560f-44cd-b975-4d50b55db9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:53.349511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c61be-560f-44cd-b975-4d50b55db9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:53.349804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c61be-560f-44cd-b975-4d50b55db9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:53.350060 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c61be-560f-44cd-b975-4d50b55db9b2 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:53.350908 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c61be-560f-44cd-b975-4d50b55db9b2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:53.351221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c61be-560f-44cd-b975-4d50b55db9b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:53.352804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c61be-560f-44cd-b975-4d50b55db9b2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:53.368725 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033076d-86fb-4769-909f-54beed969e5e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:53.378172 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 443/864] 10.222.0.42 () {56 vars in 1135 bytes} [Tue Jul 19 15:17:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:53.379686 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c61be-560f-44cd-b975-4d50b55db9b2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:53.414614 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9a2b9-0550-4304-b7bf-7114b9dff45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:53.415488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9a2b9-0550-4304-b7bf-7114b9dff45c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:53.415767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9a2b9-0550-4304-b7bf-7114b9dff45c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:53.416045 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9a2b9-0550-4304-b7bf-7114b9dff45c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:53.419898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:53.425304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9a2b9-0550-4304-b7bf-7114b9dff45c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:53.427023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9a2b9-0550-4304-b7bf-7114b9dff45c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:53.446306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3fd9a2b9-0550-4304-b7bf-7114b9dff45c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:53.446306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:53.446306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:53.446306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:53.446306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:53.446306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:53.446306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:53.446306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:53.446306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:53.446306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:53.446306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:53.446306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:53.446306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:53.446306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:53.446306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:53.447227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:53.448121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:53.448121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:53.448121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:53.448121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:53.448121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:53.448121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:53.448121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:53.448121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:53.452677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 422/865] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:53.454256 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9a2b9-0550-4304-b7bf-7114b9dff45c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:53.506576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c61be-560f-44cd-b975-4d50b55db9b2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:53.514705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:53.544303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61144cfe-f614-46cf-a1c3-ba1e8631824f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:53.545696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61144cfe-f614-46cf-a1c3-ba1e8631824f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:53.546434 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61144cfe-f614-46cf-a1c3-ba1e8631824f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:53.546916 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61144cfe-f614-46cf-a1c3-ba1e8631824f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:53.597065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9a2b9-0550-4304-b7bf-7114b9dff45c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:53.634942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61144cfe-f614-46cf-a1c3-ba1e8631824f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:53.637319 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61144cfe-f614-46cf-a1c3-ba1e8631824f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:53.648665 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396e10c-2911-4f63-9b43-697afe3df353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:53.650221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396e10c-2911-4f63-9b43-697afe3df353 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:53.650604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396e10c-2911-4f63-9b43-697afe3df353 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:53.650976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396e10c-2911-4f63-9b43-697afe3df353 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:53.652321 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396e10c-2911-4f63-9b43-697afe3df353 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:53.652732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396e10c-2911-4f63-9b43-697afe3df353 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:53.655028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396e10c-2911-4f63-9b43-697afe3df353 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:53.676335 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 423/866] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:53.678273 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396e10c-2911-4f63-9b43-697afe3df353 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:53.680044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:53.769063 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 444/867] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:53 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:17:53.769063 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8f5ea-8a4f-448b-8c99-981cf1ecf08c None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:53.769063 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8f5ea-8a4f-448b-8c99-981cf1ecf08c None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:53.769063 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8f5ea-8a4f-448b-8c99-981cf1ecf08c None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:53.772159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 445/868] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:53.773174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8f5ea-8a4f-448b-8c99-981cf1ecf08c None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:53.791571 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:53.796111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396e10c-2911-4f63-9b43-697afe3df353 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:53.831477 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867c658-4b27-4631-b58b-02f171481e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:53.832130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867c658-4b27-4631-b58b-02f171481e7b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:53.832419 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867c658-4b27-4631-b58b-02f171481e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:53.832689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867c658-4b27-4631-b58b-02f171481e7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:53.834501 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867c658-4b27-4631-b58b-02f171481e7b None admin] RBAC: Adding query filter params (role.id=c8578ede4e7041ecaeba79c255cc650b, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=60f89f6042664f91b2829d516ed7e844) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:53.834812 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867c658-4b27-4631-b58b-02f171481e7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:53.836245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867c658-4b27-4631-b58b-02f171481e7b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:53.853555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 424/869] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:53 2022] GET /identity/v3/role_assignments?user.id=60f89f6042664f91b2829d516ed7e844&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=c8578ede4e7041ecaeba79c255cc650b => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:53.854868 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10e794-d74d-4830-bd82-61e63927292c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:53.855157 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10e794-d74d-4830-bd82-61e63927292c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:53.855435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10e794-d74d-4830-bd82-61e63927292c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:53.874094 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2e10e794-d74d-4830-bd82-61e63927292c None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:53.875308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2e10e794-d74d-4830-bd82-61e63927292c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:53.897416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8f5ea-8a4f-448b-8c99-981cf1ecf08c None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:53.946282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:53.955119 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb61a1f-4a1f-4c78-9ca6-7b283fca5dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:53.956527 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb61a1f-4a1f-4c78-9ca6-7b283fca5dce None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:53.956991 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb61a1f-4a1f-4c78-9ca6-7b283fca5dce None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:53.957535 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb61a1f-4a1f-4c78-9ca6-7b283fca5dce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:53.962404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 425/870] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:53 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:17:53.971651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb61a1f-4a1f-4c78-9ca6-7b283fca5dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:53.973845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb61a1f-4a1f-4c78-9ca6-7b283fca5dce None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:53.985063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10e794-d74d-4830-bd82-61e63927292c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:53.998235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-8eb61a1f-4a1f-4c78-9ca6-7b283fca5dce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:53.998235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:53.998235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:53.998235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:53.998235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:53.998235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:53.998235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:53.998235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:53.998235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:53.998235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:53.998235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:53.998235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:53.998235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:53.998235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:53.998235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:53.999328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:54.000225 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:54.000225 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:54.000225 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:54.000225 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:54.002869 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 446/871] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:54.009021 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb61a1f-4a1f-4c78-9ca6-7b283fca5dce None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:54.110086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:54.121392 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10e794-d74d-4830-bd82-61e63927292c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:54.133260 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb61a1f-4a1f-4c78-9ca6-7b283fca5dce None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:54.160624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb9673-2a50-406d-9a8d-2eaa758be410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:54.161591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb9673-2a50-406d-9a8d-2eaa758be410 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:54.162019 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb9673-2a50-406d-9a8d-2eaa758be410 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:54.162534 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb9673-2a50-406d-9a8d-2eaa758be410 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:54.163341 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb9673-2a50-406d-9a8d-2eaa758be410 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:54.164768 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb9673-2a50-406d-9a8d-2eaa758be410 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:54.170448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa6cf3-0dbe-4366-8dc3-7a6dd7049620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:54.171120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa6cf3-0dbe-4366-8dc3-7a6dd7049620 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:54.171393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa6cf3-0dbe-4366-8dc3-7a6dd7049620 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:54.171669 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa6cf3-0dbe-4366-8dc3-7a6dd7049620 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:54.172401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa6cf3-0dbe-4366-8dc3-7a6dd7049620 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:54.172705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa6cf3-0dbe-4366-8dc3-7a6dd7049620 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:54.174766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa6cf3-0dbe-4366-8dc3-7a6dd7049620 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:54.188165 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 426/872] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Jul 19 15:17:53 2022] POST /identity/v3/services => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:17:54.191488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 447/873] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:54.196591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb9673-2a50-406d-9a8d-2eaa758be410 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:54.329256 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb9673-2a50-406d-9a8d-2eaa758be410 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:54.370752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb0de7-64d2-45b6-89b2-8c10e078113a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:54.371855 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb0de7-64d2-45b6-89b2-8c10e078113a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:54.372360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb0de7-64d2-45b6-89b2-8c10e078113a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:54.372819 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb0de7-64d2-45b6-89b2-8c10e078113a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:54.380703 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb0de7-64d2-45b6-89b2-8c10e078113a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:54.382538 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb0de7-64d2-45b6-89b2-8c10e078113a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:54.398688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-bbcb0de7-64d2-45b6-89b2-8c10e078113a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:54.398688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:54.398688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:54.398688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:54.398688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:54.398688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:54.398688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:54.398688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:54.398688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:54.398688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:54.398688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:54.398688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:54.398688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:54.398688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:54.398688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:54.399652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:54.400817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:54.400817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:54.400817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:54.400817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:54.400817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:54.400817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:54.400817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:54.400817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:54.403540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 427/874] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:54.406790 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa6cf3-0dbe-4366-8dc3-7a6dd7049620 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:54.457058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:54.523421 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa6cf3-0dbe-4366-8dc3-7a6dd7049620 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:54.562076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ff916-0397-4ad5-8626-e2ce6ab7d89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:54.562744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ff916-0397-4ad5-8626-e2ce6ab7d89a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:54.563029 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ff916-0397-4ad5-8626-e2ce6ab7d89a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:54.563308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ff916-0397-4ad5-8626-e2ce6ab7d89a None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:54.564658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ff916-0397-4ad5-8626-e2ce6ab7d89a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:54.564966 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ff916-0397-4ad5-8626-e2ce6ab7d89a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:54.566423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ff916-0397-4ad5-8626-e2ce6ab7d89a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:54.589992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 448/875] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:54.598290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb0de7-64d2-45b6-89b2-8c10e078113a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:54.609761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:54.710557 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb0de7-64d2-45b6-89b2-8c10e078113a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:54.750115 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac16e25-80d2-4ed0-8c06-40531e6e5d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:54.752258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac16e25-80d2-4ed0-8c06-40531e6e5d51 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:54.753058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac16e25-80d2-4ed0-8c06-40531e6e5d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:54.753780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac16e25-80d2-4ed0-8c06-40531e6e5d51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:54.755339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac16e25-80d2-4ed0-8c06-40531e6e5d51 None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=96faaac55bc5408685c68a6177d91971) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:54.755888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac16e25-80d2-4ed0-8c06-40531e6e5d51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:54.757641 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac16e25-80d2-4ed0-8c06-40531e6e5d51 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:54.764935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 428/876] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:54 2022] GET /identity/v3/role_assignments?user.id=96faaac55bc5408685c68a6177d91971&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:54.950921 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd006ee-0830-47c4-a570-8ce5aa7ee99e None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:54.951207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd006ee-0830-47c4-a570-8ce5aa7ee99e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:54.951483 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd006ee-0830-47c4-a570-8ce5aa7ee99e None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:54.952990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 449/877] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:54.958465 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f6280-5a9f-492c-bcf6-55519c6ef76d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:54.959002 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f6280-5a9f-492c-bcf6-55519c6ef76d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:54.959477 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f6280-5a9f-492c-bcf6-55519c6ef76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:54.977014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2b7f6280-5a9f-492c-bcf6-55519c6ef76d None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:54.978257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2b7f6280-5a9f-492c-bcf6-55519c6ef76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:55.050648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 429/878] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:17:55.074601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd006ee-0830-47c4-a570-8ce5aa7ee99e None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:55.187246 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd006ee-0830-47c4-a570-8ce5aa7ee99e None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:55.201843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:55.223845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf3607-e35f-4757-98f1-697d58d6897d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:55.224656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf3607-e35f-4757-98f1-697d58d6897d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:55.224999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf3607-e35f-4757-98f1-697d58d6897d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:55.225314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf3607-e35f-4757-98f1-697d58d6897d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:55.226024 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf3607-e35f-4757-98f1-697d58d6897d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:55.227497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf3607-e35f-4757-98f1-697d58d6897d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-23cf3607-e35f-4757-98f1-697d58d6897d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:55.240749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:55.241747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:55.242577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 450/879] 10.222.0.42 () {56 vars in 1132 bytes} [Tue Jul 19 15:17:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:55.248283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f6280-5a9f-492c-bcf6-55519c6ef76d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:55.253095 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:55.364812 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f6280-5a9f-492c-bcf6-55519c6ef76d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:55.401771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c53b1-bfad-4d0d-bc8f-1ccda7208f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:55.402717 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c53b1-bfad-4d0d-bc8f-1ccda7208f72 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:55.403152 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c53b1-bfad-4d0d-bc8f-1ccda7208f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:55.403582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c53b1-bfad-4d0d-bc8f-1ccda7208f72 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:55.404513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c53b1-bfad-4d0d-bc8f-1ccda7208f72 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:55.404969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c53b1-bfad-4d0d-bc8f-1ccda7208f72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:55.406638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c53b1-bfad-4d0d-bc8f-1ccda7208f72 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:55.412627 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 430/880] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Jul 19 15:17:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:55.418511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf3607-e35f-4757-98f1-697d58d6897d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:55.526701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf3607-e35f-4757-98f1-697d58d6897d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:55.564209 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfe03c-c3c8-4d2a-80d2-d1a66463b4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:55.566094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfe03c-c3c8-4d2a-80d2-d1a66463b4cc None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:55.567099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfe03c-c3c8-4d2a-80d2-d1a66463b4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:55.567732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfe03c-c3c8-4d2a-80d2-d1a66463b4cc None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:55.568890 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdfe03c-c3c8-4d2a-80d2-d1a66463b4cc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:55.569640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdfe03c-c3c8-4d2a-80d2-d1a66463b4cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:55.571266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdfe03c-c3c8-4d2a-80d2-d1a66463b4cc None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:55.578045 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 451/881] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Jul 19 15:17:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:55.584303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c53b1-bfad-4d0d-bc8f-1ccda7208f72 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:55.673634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:55.723920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c53b1-bfad-4d0d-bc8f-1ccda7208f72 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:55.771766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5cf07-e999-4894-a60b-112f4a35d607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:55.772442 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5cf07-e999-4894-a60b-112f4a35d607 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:55.772720 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5cf07-e999-4894-a60b-112f4a35d607 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:55.773677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5cf07-e999-4894-a60b-112f4a35d607 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:55.773856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5cf07-e999-4894-a60b-112f4a35d607 None admin] RBAC: Adding query filter params (interface=public, service_id=cacc359d8ddb49c09c09dd58d45af8b1, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:55.774121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5cf07-e999-4894-a60b-112f4a35d607 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:55.775738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5cf07-e999-4894-a60b-112f4a35d607 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:55.783075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 431/882] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Jul 19 15:17:55 2022] GET /identity/v3/endpoints?service_id=cacc359d8ddb49c09c09dd58d45af8b1&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:55.788783 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfe03c-c3c8-4d2a-80d2-d1a66463b4cc None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:55.829634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:55.912068 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfe03c-c3c8-4d2a-80d2-d1a66463b4cc None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:55.948272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ec1a3-74e5-4a63-9812-28e47623fb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:55.949298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ec1a3-74e5-4a63-9812-28e47623fb64 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:55.949812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ec1a3-74e5-4a63-9812-28e47623fb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:55.950304 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ec1a3-74e5-4a63-9812-28e47623fb64 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:55.951208 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ec1a3-74e5-4a63-9812-28e47623fb64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:55.952852 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ec1a3-74e5-4a63-9812-28e47623fb64 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:55.959373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 452/883] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Jul 19 15:17:55 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:17:56.142596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dc9b2-b03a-44cb-87b9-357f03ad1a51 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:56.143122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dc9b2-b03a-44cb-87b9-357f03ad1a51 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:56.143122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dc9b2-b03a-44cb-87b9-357f03ad1a51 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:56.144728 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 432/884] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:56.150540 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c09a1-f3d9-4114-8146-080b78aaa294 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:56.150984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c09a1-f3d9-4114-8146-080b78aaa294 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:56.151080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c09a1-f3d9-4114-8146-080b78aaa294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:56.169999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-195c09a1-f3d9-4114-8146-080b78aaa294 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:56.171241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-195c09a1-f3d9-4114-8146-080b78aaa294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:56.248383 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 453/885] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:17:56.271277 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5dc9b2-b03a-44cb-87b9-357f03ad1a51 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:56.381283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5dc9b2-b03a-44cb-87b9-357f03ad1a51 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:56.412832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:56.419334 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ddc1b-bcd0-4580-8ccc-d12a423bcbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:56.420403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ddc1b-bcd0-4580-8ccc-d12a423bcbaf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:56.420908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ddc1b-bcd0-4580-8ccc-d12a423bcbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:56.421440 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ddc1b-bcd0-4580-8ccc-d12a423bcbaf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:56.433221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ddc1b-bcd0-4580-8ccc-d12a423bcbaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c42ddc1b-bcd0-4580-8ccc-d12a423bcbaf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:17:56.441814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:56.442744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:56.443597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:56.443597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:56.443597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:56.443597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:56.443597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:17:56.443597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:17:56.443597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:17:56.443597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:17:56.443597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:17:56.443597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:17:56.443597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 15:17:56.443597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:56.446321 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 433/886] 10.222.0.42 () {56 vars in 1120 bytes} [Tue Jul 19 15:17:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:56.454836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c09a1-f3d9-4114-8146-080b78aaa294 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:56.481049 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:56.487733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd097d-9b22-42a1-97aa-8ababdef0c04 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:56.488444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd097d-9b22-42a1-97aa-8ababdef0c04 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:56.488937 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd097d-9b22-42a1-97aa-8ababdef0c04 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:56.491370 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 434/887] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:17:56.495920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0d515-fa5f-4192-bf27-aefa9d630095 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:56.496458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0d515-fa5f-4192-bf27-aefa9d630095 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:56.496952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0d515-fa5f-4192-bf27-aefa9d630095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:56.519746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-dde0d515-fa5f-4192-bf27-aefa9d630095 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:56.521853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-dde0d515-fa5f-4192-bf27-aefa9d630095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:56.564556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c09a1-f3d9-4114-8146-080b78aaa294 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:56.616766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 435/888] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:17:56.625726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45269603-0aab-47f1-8afd-d75db325c130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:56.626563 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45269603-0aab-47f1-8afd-d75db325c130 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:56.627100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45269603-0aab-47f1-8afd-d75db325c130 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:56.627454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45269603-0aab-47f1-8afd-d75db325c130 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:56.628293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45269603-0aab-47f1-8afd-d75db325c130 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:56.628672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45269603-0aab-47f1-8afd-d75db325c130 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:56.630229 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45269603-0aab-47f1-8afd-d75db325c130 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:56.638016 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0d515-fa5f-4192-bf27-aefa9d630095 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:56.639471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 454/889] 10.222.0.42 () {56 vars in 1115 bytes} [Tue Jul 19 15:17:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:56.645770 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45269603-0aab-47f1-8afd-d75db325c130 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:56.748558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0d515-fa5f-4192-bf27-aefa9d630095 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:56.759270 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:56.778162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:56.780157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45269603-0aab-47f1-8afd-d75db325c130 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:56.799119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa11203-b7a1-43e7-8bf3-cef15cec1182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:56.800494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa11203-b7a1-43e7-8bf3-cef15cec1182 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:56.800854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa11203-b7a1-43e7-8bf3-cef15cec1182 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:56.801373 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa11203-b7a1-43e7-8bf3-cef15cec1182 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:56.802242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa11203-b7a1-43e7-8bf3-cef15cec1182 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:56.804324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa11203-b7a1-43e7-8bf3-cef15cec1182 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-eaa11203-b7a1-43e7-8bf3-cef15cec1182 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:56.820149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:56.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:56.826797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:56.826797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:56.826797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:56.826797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:56.826797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:56.826797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:56.826797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:56.826797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:56.826797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:56.826797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:56.826797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:17:56.826797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:56.826797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 15:17:56.826797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:56.826797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 436/890] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:17:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:56.827585 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa11203-b7a1-43e7-8bf3-cef15cec1182 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:56.830201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa605e1d-1297-42b7-a656-1fa3a3f09b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:56.831058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa605e1d-1297-42b7-a656-1fa3a3f09b2c None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:56.831395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa605e1d-1297-42b7-a656-1fa3a3f09b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:56.831666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa605e1d-1297-42b7-a656-1fa3a3f09b2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:56.843630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa605e1d-1297-42b7-a656-1fa3a3f09b2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:56.845786 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa605e1d-1297-42b7-a656-1fa3a3f09b2c None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:56.865351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-aa605e1d-1297-42b7-a656-1fa3a3f09b2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:56.865351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:56.865351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:56.865351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:56.865351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:56.865351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:56.865351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:56.865351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:56.865351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:56.865351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:56.865351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:56.865351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:56.865351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:56.865351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:56.865351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:56.866293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:56.867182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:56.867182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:56.867182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:56.867182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:56.867182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 455/891] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:56.872238 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa605e1d-1297-42b7-a656-1fa3a3f09b2c None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:56.942154 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa11203-b7a1-43e7-8bf3-cef15cec1182 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:56.978989 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e7127-95bf-4bff-9670-d9137bc59990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:56.980066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e7127-95bf-4bff-9670-d9137bc59990 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:56.980570 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e7127-95bf-4bff-9670-d9137bc59990 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:56.981049 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e7127-95bf-4bff-9670-d9137bc59990 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:56.981985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e7127-95bf-4bff-9670-d9137bc59990 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:56.982495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e7127-95bf-4bff-9670-d9137bc59990 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:56.984040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e7127-95bf-4bff-9670-d9137bc59990 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:56.994326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa605e1d-1297-42b7-a656-1fa3a3f09b2c None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:56.994860 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 437/892] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:17:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:56.995886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e7127-95bf-4bff-9670-d9137bc59990 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:57.028096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8754a-3bf4-41c4-9c6e-9d6693e48e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:57.028675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8754a-3bf4-41c4-9c6e-9d6693e48e16 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:57.028967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8754a-3bf4-41c4-9c6e-9d6693e48e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:57.029324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8754a-3bf4-41c4-9c6e-9d6693e48e16 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:57.030063 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8754a-3bf4-41c4-9c6e-9d6693e48e16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:57.030376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8754a-3bf4-41c4-9c6e-9d6693e48e16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:57.031818 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8754a-3bf4-41c4-9c6e-9d6693e48e16 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:57.046099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:57.046806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 456/893] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:57.050658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:57.053663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8754a-3bf4-41c4-9c6e-9d6693e48e16 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:57.114892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e7127-95bf-4bff-9670-d9137bc59990 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:57.152017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711b60b-ea4f-446c-813c-65a54f6e6246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:57.153651 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711b60b-ea4f-446c-813c-65a54f6e6246 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:57.154135 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711b60b-ea4f-446c-813c-65a54f6e6246 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:57.154619 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711b60b-ea4f-446c-813c-65a54f6e6246 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:57.155691 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711b60b-ea4f-446c-813c-65a54f6e6246 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:57.156251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711b60b-ea4f-446c-813c-65a54f6e6246 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:57.159035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711b60b-ea4f-446c-813c-65a54f6e6246 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:57.169770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 438/894] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:17:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:57.171010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711b60b-ea4f-446c-813c-65a54f6e6246 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:57.180332 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8754a-3bf4-41c4-9c6e-9d6693e48e16 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:57.224396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cbf31d-1c55-40cc-9e83-64d2df2e2194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:57.225383 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbf31d-1c55-40cc-9e83-64d2df2e2194 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:57.225963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbf31d-1c55-40cc-9e83-64d2df2e2194 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:57.226246 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbf31d-1c55-40cc-9e83-64d2df2e2194 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:57.235736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cbf31d-1c55-40cc-9e83-64d2df2e2194 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:57.237387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cbf31d-1c55-40cc-9e83-64d2df2e2194 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-73cbf31d-1c55-40cc-9e83-64d2df2e2194 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:17:57.257046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:57.258228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:57.259180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:57.259180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:57.259180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:17:57.259180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:17:57.259180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:17:57.259180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:17:57.259180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:17:57.259180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:17:57.259180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 15:17:57.259180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:57.259180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 457/895] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Jul 19 15:17:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:57.259771 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d45fb-fb49-4f98-a17c-0b69453a1d60 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:57.259771 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d45fb-fb49-4f98-a17c-0b69453a1d60 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:57.259901 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d45fb-fb49-4f98-a17c-0b69453a1d60 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:57.261300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 458/896] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:57.264410 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d45fb-fb49-4f98-a17c-0b69453a1d60 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:57.286966 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711b60b-ea4f-446c-813c-65a54f6e6246 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:57.300698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:57.323856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68588c63-83e1-47ad-8983-c12449b0b54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:57.324508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68588c63-83e1-47ad-8983-c12449b0b54e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:57.324787 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68588c63-83e1-47ad-8983-c12449b0b54e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:57.325061 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68588c63-83e1-47ad-8983-c12449b0b54e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:57.325885 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68588c63-83e1-47ad-8983-c12449b0b54e None admin] RBAC: Adding query filter params (interface=public, service_id=c4d6943e8a7242e58903ab2bac593ce2, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:57.326195 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68588c63-83e1-47ad-8983-c12449b0b54e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:57.327545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68588c63-83e1-47ad-8983-c12449b0b54e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:57.329867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:57.334567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 439/897] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Jul 19 15:17:57 2022] GET /identity/v3/endpoints?service_id=c4d6943e8a7242e58903ab2bac593ce2&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:57.335762 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77500e66-2be9-4d92-b09c-cf8c4b064098 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:57.336049 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77500e66-2be9-4d92-b09c-cf8c4b064098 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:57.336322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77500e66-2be9-4d92-b09c-cf8c4b064098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:57.353262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-77500e66-2be9-4d92-b09c-cf8c4b064098 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:57.354204 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-77500e66-2be9-4d92-b09c-cf8c4b064098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:57.378061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d45fb-fb49-4f98-a17c-0b69453a1d60 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:57.414013 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fddfe7-d501-401d-bddd-bddfd43cab72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:57.414776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fddfe7-d501-401d-bddd-bddfd43cab72 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:57.415067 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fddfe7-d501-401d-bddd-bddfd43cab72 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:57.415353 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fddfe7-d501-401d-bddd-bddfd43cab72 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:57.416201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fddfe7-d501-401d-bddd-bddfd43cab72 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:57.416517 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fddfe7-d501-401d-bddd-bddfd43cab72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:57.417997 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fddfe7-d501-401d-bddd-bddfd43cab72 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:57.426767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 440/898] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:17:57.437356 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77500e66-2be9-4d92-b09c-cf8c4b064098 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:57.437886 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 459/899] 10.222.0.42 () {56 vars in 1149 bytes} [Tue Jul 19 15:17:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:57.443395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fddfe7-d501-401d-bddd-bddfd43cab72 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:57.542767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77500e66-2be9-4d92-b09c-cf8c4b064098 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:57.550594 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fddfe7-d501-401d-bddd-bddfd43cab72 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:57.551563 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:57.568613 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:57.581697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff169d-5c66-4946-8fd3-a8dc5947b1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:57.582473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff169d-5c66-4946-8fd3-a8dc5947b1bc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:57.582820 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff169d-5c66-4946-8fd3-a8dc5947b1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:57.583165 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff169d-5c66-4946-8fd3-a8dc5947b1bc None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:57.583917 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff169d-5c66-4946-8fd3-a8dc5947b1bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:57.585374 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff169d-5c66-4946-8fd3-a8dc5947b1bc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:57.588549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065e486-5479-4c00-b531-f6b13754ba16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:57.589230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065e486-5479-4c00-b531-f6b13754ba16 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:57.589518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065e486-5479-4c00-b531-f6b13754ba16 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:57.589793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065e486-5479-4c00-b531-f6b13754ba16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:57.601505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 441/900] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Jul 19 15:17:57 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:17:57.613276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff169d-5c66-4946-8fd3-a8dc5947b1bc None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:57.670975 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065e486-5479-4c00-b531-f6b13754ba16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:57.672457 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065e486-5479-4c00-b531-f6b13754ba16 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:57.742092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff169d-5c66-4946-8fd3-a8dc5947b1bc None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:57.790741 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830dab4-db7f-4a5d-aed4-4a66fff28af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:57.791953 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830dab4-db7f-4a5d-aed4-4a66fff28af8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:57.791953 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830dab4-db7f-4a5d-aed4-4a66fff28af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:57.792123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830dab4-db7f-4a5d-aed4-4a66fff28af8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:57.792723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830dab4-db7f-4a5d-aed4-4a66fff28af8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:57.794231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830dab4-db7f-4a5d-aed4-4a66fff28af8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:57.799020 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 460/901] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:17:57 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:17:57.804524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065e486-5479-4c00-b531-f6b13754ba16 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9830dab4-db7f-4a5d-aed4-4a66fff28af8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:57.807245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:57.808249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:57.809122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:57.809122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:57.809122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:57.809122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:57.809122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:57.809122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:57.809122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:57.809122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:57.809122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:57.809122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:57.809122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:17:57.809122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:57.809122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 15:17:57.809122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:57.816805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 442/902] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Jul 19 15:17:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:57.816956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830dab4-db7f-4a5d-aed4-4a66fff28af8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:57.825039 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:57.868017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:57.923829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830dab4-db7f-4a5d-aed4-4a66fff28af8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:57.932345 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065e486-5479-4c00-b531-f6b13754ba16 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:57.960898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc370fb-5084-433a-a7bb-aecf5fc1fc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:57.961988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc370fb-5084-433a-a7bb-aecf5fc1fc4b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:57.962487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc370fb-5084-433a-a7bb-aecf5fc1fc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:57.962960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc370fb-5084-433a-a7bb-aecf5fc1fc4b None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:57.963939 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc370fb-5084-433a-a7bb-aecf5fc1fc4b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:57.964450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc370fb-5084-433a-a7bb-aecf5fc1fc4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:57.966068 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc370fb-5084-433a-a7bb-aecf5fc1fc4b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:57.971833 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 443/903] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Jul 19 15:17:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:57.980781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc370fb-5084-433a-a7bb-aecf5fc1fc4b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:57.980925 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c9c4e-d2ef-4207-aa6c-19d9e2999c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:57.981194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c9c4e-d2ef-4207-aa6c-19d9e2999c2e None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:57.985177 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c9c4e-d2ef-4207-aa6c-19d9e2999c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:57.985177 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c9c4e-d2ef-4207-aa6c-19d9e2999c2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:57.995127 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c9c4e-d2ef-4207-aa6c-19d9e2999c2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:57.997123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c9c4e-d2ef-4207-aa6c-19d9e2999c2e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:58.020009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-2b8c9c4e-d2ef-4207-aa6c-19d9e2999c2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:58.020009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:58.020009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:58.020009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:58.020009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:58.020009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:58.020009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:58.020009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:58.020009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:58.020009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:58.020009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:58.020009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:58.020009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:17:58.020009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:17:58.020009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:58.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:58.021912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:17:58.021912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:17:58.021912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:17:58.021912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:58.025230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 461/904] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:58.029121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c9c4e-d2ef-4207-aa6c-19d9e2999c2e None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:58.091056 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:58.095880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc370fb-5084-433a-a7bb-aecf5fc1fc4b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:58.114026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:58.134184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433fd6c-81f3-4316-ac76-ee36b09db18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:58.134856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433fd6c-81f3-4316-ac76-ee36b09db18d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:58.135138 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433fd6c-81f3-4316-ac76-ee36b09db18d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:58.135579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433fd6c-81f3-4316-ac76-ee36b09db18d None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:58.136113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433fd6c-81f3-4316-ac76-ee36b09db18d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:58.136419 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433fd6c-81f3-4316-ac76-ee36b09db18d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:58.137803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433fd6c-81f3-4316-ac76-ee36b09db18d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:58.149198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 444/905] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Jul 19 15:17:57 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:58.149198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b36d9a-da3f-4d95-971e-dbd1b7b1bdff None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:58.149198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b36d9a-da3f-4d95-971e-dbd1b7b1bdff None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:58.149198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b36d9a-da3f-4d95-971e-dbd1b7b1bdff None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:58.149198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 445/906] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:58.157350 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c9c4e-d2ef-4207-aa6c-19d9e2999c2e None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:58.161320 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091ff46-84f5-4c81-9c06-90fa77e30252 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:58.162182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091ff46-84f5-4c81-9c06-90fa77e30252 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:58.162595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091ff46-84f5-4c81-9c06-90fa77e30252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:58.186709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6091ff46-84f5-4c81-9c06-90fa77e30252 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:58.188802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6091ff46-84f5-4c81-9c06-90fa77e30252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:58.198424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa51b3f-1943-49a2-9100-7faf45bd256d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:58.199117 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa51b3f-1943-49a2-9100-7faf45bd256d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:58.199470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa51b3f-1943-49a2-9100-7faf45bd256d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:58.199754 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa51b3f-1943-49a2-9100-7faf45bd256d None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:58.200557 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa51b3f-1943-49a2-9100-7faf45bd256d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:58.200877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa51b3f-1943-49a2-9100-7faf45bd256d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:58.202355 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa51b3f-1943-49a2-9100-7faf45bd256d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:58.221362 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 462/907] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:58.223259 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa51b3f-1943-49a2-9100-7faf45bd256d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:58.286541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 446/908] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:17:58.309211 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091ff46-84f5-4c81-9c06-90fa77e30252 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:58.363534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa51b3f-1943-49a2-9100-7faf45bd256d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:58.397686 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:58.412323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f487a4-b0d2-4a60-a4d3-00cd09413d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:58.413126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f487a4-b0d2-4a60-a4d3-00cd09413d83 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:58.413430 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f487a4-b0d2-4a60-a4d3-00cd09413d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:58.413708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f487a4-b0d2-4a60-a4d3-00cd09413d83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:58.424113 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f487a4-b0d2-4a60-a4d3-00cd09413d83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:58.425948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:58.427674 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f487a4-b0d2-4a60-a4d3-00cd09413d83 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:58.436332 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091ff46-84f5-4c81-9c06-90fa77e30252 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:58.449261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-d2f487a4-b0d2-4a60-a4d3-00cd09413d83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:58.449261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:58.449261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:58.449261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:58.449261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:58.449261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:58.449261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:58.449261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:58.449261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:58.449261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:58.449261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:58.449261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:58.449261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:17:58.449261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:17:58.449261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:58.450193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:58.451116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:17:58.451116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:17:58.451116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:17:58.451116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:17:58.451116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:17:58.451116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:17:58.451116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:17:58.451116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:17:58.454158 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 463/909] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:17:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:17:58.461188 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f487a4-b0d2-4a60-a4d3-00cd09413d83 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:58.480422 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a894c-0e82-4563-9cc8-e7c79155e145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:58.481435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a894c-0e82-4563-9cc8-e7c79155e145 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:58.481731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a894c-0e82-4563-9cc8-e7c79155e145 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:58.482009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a894c-0e82-4563-9cc8-e7c79155e145 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:58.482667 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a894c-0e82-4563-9cc8-e7c79155e145 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:58.484152 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a894c-0e82-4563-9cc8-e7c79155e145 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-cf1a894c-0e82-4563-9cc8-e7c79155e145 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:17:58.497924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:17:58.499080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:17:58.505242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 447/910] 10.222.0.42 () {56 vars in 1132 bytes} [Tue Jul 19 15:17:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:17:58.509823 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a894c-0e82-4563-9cc8-e7c79155e145 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:58.589991 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f487a4-b0d2-4a60-a4d3-00cd09413d83 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:58.629702 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a894c-0e82-4563-9cc8-e7c79155e145 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:58.630041 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6bc7c-6e6f-468d-aa4f-8bb806c5aa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:58.630918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6bc7c-6e6f-468d-aa4f-8bb806c5aa27 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:58.631413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6bc7c-6e6f-468d-aa4f-8bb806c5aa27 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:58.631904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6bc7c-6e6f-468d-aa4f-8bb806c5aa27 None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:58.632898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6bc7c-6e6f-468d-aa4f-8bb806c5aa27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:58.633432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6bc7c-6e6f-468d-aa4f-8bb806c5aa27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:58.635068 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6bc7c-6e6f-468d-aa4f-8bb806c5aa27 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:58.652823 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 464/911] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:17:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:58.654497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6bc7c-6e6f-468d-aa4f-8bb806c5aa27 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:58.667265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c36582-b555-4bc1-a998-ec9f5f445af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:58.667947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c36582-b555-4bc1-a998-ec9f5f445af3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:58.668289 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c36582-b555-4bc1-a998-ec9f5f445af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:58.668573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c36582-b555-4bc1-a998-ec9f5f445af3 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:58.669407 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c36582-b555-4bc1-a998-ec9f5f445af3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:58.669660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c36582-b555-4bc1-a998-ec9f5f445af3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:58.671027 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c36582-b555-4bc1-a998-ec9f5f445af3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:58.680471 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 448/912] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Jul 19 15:17:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:58.684453 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c36582-b555-4bc1-a998-ec9f5f445af3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:58.716757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:58.722422 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:58.789333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6bc7c-6e6f-468d-aa4f-8bb806c5aa27 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:58.813582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c36582-b555-4bc1-a998-ec9f5f445af3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:58.830578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af95e44-47e0-4efd-9aaf-9a3eeb302907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:58.831351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af95e44-47e0-4efd-9aaf-9a3eeb302907 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:58.831636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af95e44-47e0-4efd-9aaf-9a3eeb302907 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:58.831912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af95e44-47e0-4efd-9aaf-9a3eeb302907 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:58.832749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af95e44-47e0-4efd-9aaf-9a3eeb302907 None admin] RBAC: Adding query filter params (role.id=d7b232d8f46040ba8eab9538a784e61f, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=60f89f6042664f91b2829d516ed7e844) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:58.833059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af95e44-47e0-4efd-9aaf-9a3eeb302907 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:58.834534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af95e44-47e0-4efd-9aaf-9a3eeb302907 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:58.841944 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 465/913] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:17:58 2022] GET /identity/v3/role_assignments?user.id=60f89f6042664f91b2829d516ed7e844&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=d7b232d8f46040ba8eab9538a784e61f => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:17:58.856439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea642e-4862-41d2-9511-860d88ff15d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:58.857164 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea642e-4862-41d2-9511-860d88ff15d0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:58.857448 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea642e-4862-41d2-9511-860d88ff15d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:58.857724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea642e-4862-41d2-9511-860d88ff15d0 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:58.858453 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea642e-4862-41d2-9511-860d88ff15d0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:17:58.858761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea642e-4862-41d2-9511-860d88ff15d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:58.860186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea642e-4862-41d2-9511-860d88ff15d0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:58.866567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 449/914] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Jul 19 15:17:58 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:17:58.872522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af95e44-47e0-4efd-9aaf-9a3eeb302907 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:58.990751 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af95e44-47e0-4efd-9aaf-9a3eeb302907 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:17:59.006269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:59.028956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da588a0f-6b68-428e-b6f0-cb5e4cc43bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:17:59.029659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da588a0f-6b68-428e-b6f0-cb5e4cc43bcf None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:59.029938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da588a0f-6b68-428e-b6f0-cb5e4cc43bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:59.030212 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da588a0f-6b68-428e-b6f0-cb5e4cc43bcf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:59.030955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da588a0f-6b68-428e-b6f0-cb5e4cc43bcf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:17:59.032425 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da588a0f-6b68-428e-b6f0-cb5e4cc43bcf None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:17:59.053595 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 466/915] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Jul 19 15:17:58 2022] POST /identity/v3/endpoints => generated 347 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:17:59.746898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447bea7-42e5-4fae-8095-a29c01390b4d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:59.747445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447bea7-42e5-4fae-8095-a29c01390b4d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:59.747445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447bea7-42e5-4fae-8095-a29c01390b4d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:59.749068 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 450/916] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:17:59.754601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacc747-da4e-4ea7-9479-1ef18c321c44 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:17:59.755045 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacc747-da4e-4ea7-9479-1ef18c321c44 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:17:59.755134 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacc747-da4e-4ea7-9479-1ef18c321c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:17:59.772323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-0aacc747-da4e-4ea7-9479-1ef18c321c44 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:17:59.773531 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0aacc747-da4e-4ea7-9479-1ef18c321c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:17:59.850216 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 467/917] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:17:59 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:17:59.872459 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447bea7-42e5-4fae-8095-a29c01390b4d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:17:59.966610 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:17:59.984333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447bea7-42e5-4fae-8095-a29c01390b4d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:00.020972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57922604-c59d-41c1-bd22-0561b81202b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:00.021755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57922604-c59d-41c1-bd22-0561b81202b7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:00.022038 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57922604-c59d-41c1-bd22-0561b81202b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:00.022315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57922604-c59d-41c1-bd22-0561b81202b7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:00.023000 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57922604-c59d-41c1-bd22-0561b81202b7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:00.024430 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57922604-c59d-41c1-bd22-0561b81202b7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-57922604-c59d-41c1-bd22-0561b81202b7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:00.037457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:00.038562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:00.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:00.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:00.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:00.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:00.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:00.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:00.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:00.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:00.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:00.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:00.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:18:00.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:00.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 15:18:00.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:00.039474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 451/918] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:17:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:00.045347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacc747-da4e-4ea7-9479-1ef18c321c44 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:00.145238 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:00.152630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacc747-da4e-4ea7-9479-1ef18c321c44 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:00.190099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f87f0-2eb8-45fa-9a99-af1e213bca4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:00.190775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f87f0-2eb8-45fa-9a99-af1e213bca4e None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:00.191058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f87f0-2eb8-45fa-9a99-af1e213bca4e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:00.191340 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f87f0-2eb8-45fa-9a99-af1e213bca4e None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:00.192066 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f87f0-2eb8-45fa-9a99-af1e213bca4e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:00.192375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f87f0-2eb8-45fa-9a99-af1e213bca4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:00.193786 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f87f0-2eb8-45fa-9a99-af1e213bca4e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:00.199003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 468/919] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:18:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:00.204427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57922604-c59d-41c1-bd22-0561b81202b7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:00.299404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d36c9-5326-47fc-bd87-410af6001153 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:00.299601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d36c9-5326-47fc-bd87-410af6001153 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:00.299862 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d36c9-5326-47fc-bd87-410af6001153 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:00.301673 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 469/920] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:18:00.306237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8ef6b-66dd-4bbe-bf2f-132992c80fc3 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:00.306522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8ef6b-66dd-4bbe-bf2f-132992c80fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:00.306829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8ef6b-66dd-4bbe-bf2f-132992c80fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:00.316279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57922604-c59d-41c1-bd22-0561b81202b7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:00.324716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-d9c8ef6b-66dd-4bbe-bf2f-132992c80fc3 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:00.325944 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d9c8ef6b-66dd-4bbe-bf2f-132992c80fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:00.353801 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cceed3-7510-4658-af85-e8799bc73032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:00.354763 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cceed3-7510-4658-af85-e8799bc73032 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:00.355285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cceed3-7510-4658-af85-e8799bc73032 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:00.355805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cceed3-7510-4658-af85-e8799bc73032 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:00.356798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cceed3-7510-4658-af85-e8799bc73032 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:00.357317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cceed3-7510-4658-af85-e8799bc73032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:00.359179 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cceed3-7510-4658-af85-e8799bc73032 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:00.370015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 452/921] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Jul 19 15:18:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:00.371557 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cceed3-7510-4658-af85-e8799bc73032 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:00.389317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:00.401274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 470/922] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:18:00.418190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8ef6b-66dd-4bbe-bf2f-132992c80fc3 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:00.484508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cceed3-7510-4658-af85-e8799bc73032 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:00.508186 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:00.521772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8ef6b-66dd-4bbe-bf2f-132992c80fc3 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:00.524480 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35ec8a-55d7-4f49-a929-9603af03abe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:00.525782 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35ec8a-55d7-4f49-a929-9603af03abe4 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:00.526288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35ec8a-55d7-4f49-a929-9603af03abe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:00.526754 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35ec8a-55d7-4f49-a929-9603af03abe4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:00.527725 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35ec8a-55d7-4f49-a929-9603af03abe4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:00.529308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35ec8a-55d7-4f49-a929-9603af03abe4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:00.555699 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 453/923] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Jul 19 15:18:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:18:00.566384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd89c1-8e7b-44e2-a24d-301c4d039f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:00.567075 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd89c1-8e7b-44e2-a24d-301c4d039f86 None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:00.567358 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd89c1-8e7b-44e2-a24d-301c4d039f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:00.567629 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd89c1-8e7b-44e2-a24d-301c4d039f86 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:00.568377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd89c1-8e7b-44e2-a24d-301c4d039f86 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:00.569899 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd89c1-8e7b-44e2-a24d-301c4d039f86 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:00.587467 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 471/924] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Jul 19 15:18:00 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:18:01.003658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff754a61-a431-46a5-9163-25d55b82f4cf None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:01.004173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff754a61-a431-46a5-9163-25d55b82f4cf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:01.004279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff754a61-a431-46a5-9163-25d55b82f4cf None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:01.005882 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 454/925] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:01.011017 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d6c0d-bc95-48cf-9d9b-2ffa103a39fc None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:01.011454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d6c0d-bc95-48cf-9d9b-2ffa103a39fc None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:01.011638 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d6c0d-bc95-48cf-9d9b-2ffa103a39fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:01.030951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-4e6d6c0d-bc95-48cf-9d9b-2ffa103a39fc None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:01.032750 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-4e6d6c0d-bc95-48cf-9d9b-2ffa103a39fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:01.110254 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 472/926] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:01.132534 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff754a61-a431-46a5-9163-25d55b82f4cf None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:01.183138 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939b1a2-d5b4-4bb3-b5f2-317a0838dde5 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:01.183785 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939b1a2-d5b4-4bb3-b5f2-317a0838dde5 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:01.184269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939b1a2-d5b4-4bb3-b5f2-317a0838dde5 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:01.186154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 473/927] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:18:01.190833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3898d-5921-4765-a750-96a5ed61f84b None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:01.191388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3898d-5921-4765-a750-96a5ed61f84b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:01.191868 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3898d-5921-4765-a750-96a5ed61f84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:01.208411 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:01.212723 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-eac3898d-5921-4765-a750-96a5ed61f84b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:01.214201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-eac3898d-5921-4765-a750-96a5ed61f84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:01.254294 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff754a61-a431-46a5-9163-25d55b82f4cf None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:01.295407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:01.300603 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde80ee-57ad-45f2-889f-e3d6256947f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:01.301766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 474/928] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:01.305499 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde80ee-57ad-45f2-889f-e3d6256947f7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:01.305957 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde80ee-57ad-45f2-889f-e3d6256947f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:01.306433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde80ee-57ad-45f2-889f-e3d6256947f7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:01.319234 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde80ee-57ad-45f2-889f-e3d6256947f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:18:01.324738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3898d-5921-4765-a750-96a5ed61f84b None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-dbde80ee-57ad-45f2-889f-e3d6256947f7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:18:01.327726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:01.328684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:01.329642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:01.329642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:01.329642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:01.329642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:01.329642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:18:01.329642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:18:01.329642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:18:01.329642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:18:01.329642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:18:01.329642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:18:01.329642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 15:18:01.329642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:01.329642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 455/929] 10.222.0.42 () {56 vars in 1120 bytes} [Tue Jul 19 15:18:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:01.334705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde80ee-57ad-45f2-889f-e3d6256947f7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:01.433574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3898d-5921-4765-a750-96a5ed61f84b None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:01.449487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde80ee-57ad-45f2-889f-e3d6256947f7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:01.470014 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ae7bd-45f8-4e81-badc-bb75d1897aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:01.471167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ae7bd-45f8-4e81-badc-bb75d1897aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:01.471668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ae7bd-45f8-4e81-badc-bb75d1897aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:01.472106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ae7bd-45f8-4e81-badc-bb75d1897aa0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:01.472924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ae7bd-45f8-4e81-badc-bb75d1897aa0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:01.474470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ae7bd-45f8-4e81-badc-bb75d1897aa0 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:01.479099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:01.488491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81bdc0-fb7c-4077-b277-e499a286b1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-f35ae7bd-45f8-4e81-badc-bb75d1897aa0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:01.489128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:01.490155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:01.491028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:01.491028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:01.491028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:01.491028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:01.491028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:01.491028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:01.491028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:01.491028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:01.491028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:01.491028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:01.491028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:18:01.491028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:01.491028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 15:18:01.491028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:01.492600 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81bdc0-fb7c-4077-b277-e499a286b1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:01.493091 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81bdc0-fb7c-4077-b277-e499a286b1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:01.493616 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81bdc0-fb7c-4077-b277-e499a286b1d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:01.494590 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81bdc0-fb7c-4077-b277-e499a286b1d4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:01.495104 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81bdc0-fb7c-4077-b277-e499a286b1d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:01.495412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 475/930] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:18:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:18:01.496239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ae7bd-45f8-4e81-badc-bb75d1897aa0 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:01.497107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81bdc0-fb7c-4077-b277-e499a286b1d4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:01.506646 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 456/931] 10.222.0.42 () {56 vars in 1115 bytes} [Tue Jul 19 15:18:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:01.514344 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81bdc0-fb7c-4077-b277-e499a286b1d4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:01.568670 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:01.611687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ae7bd-45f8-4e81-badc-bb75d1897aa0 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:01.647130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81bdc0-fb7c-4077-b277-e499a286b1d4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:01.649375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0f0b5-4e37-4194-99a8-5db04a3cf534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:01.650607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0f0b5-4e37-4194-99a8-5db04a3cf534 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:01.650889 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0f0b5-4e37-4194-99a8-5db04a3cf534 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:01.651314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0f0b5-4e37-4194-99a8-5db04a3cf534 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:01.652171 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0f0b5-4e37-4194-99a8-5db04a3cf534 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:01.652641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0f0b5-4e37-4194-99a8-5db04a3cf534 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:01.654190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0f0b5-4e37-4194-99a8-5db04a3cf534 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:01.660107 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 476/932] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Jul 19 15:18:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:01.665700 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0f0b5-4e37-4194-99a8-5db04a3cf534 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:01.685669 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bce58-0f12-4864-a5a2-ed2d8e99a5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:01.686787 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bce58-0f12-4864-a5a2-ed2d8e99a5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:01.687217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bce58-0f12-4864-a5a2-ed2d8e99a5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:01.687637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bce58-0f12-4864-a5a2-ed2d8e99a5e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:01.699850 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4bce58-0f12-4864-a5a2-ed2d8e99a5e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:01.701577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4bce58-0f12-4864-a5a2-ed2d8e99a5e5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:01.720524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-5b4bce58-0f12-4864-a5a2-ed2d8e99a5e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:01.720524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:01.720524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:01.720524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:01.720524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:01.720524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:01.720524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:01.720524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:01.720524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:01.720524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:01.720524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:01.720524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:01.720524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:18:01.720524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:18:01.720524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:01.721568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:01.722469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:18:01.722469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:18:01.722469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:01.722469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:01.724797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 457/933] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:01.731078 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bce58-0f12-4864-a5a2-ed2d8e99a5e5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:01.786845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0f0b5-4e37-4194-99a8-5db04a3cf534 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:01.792941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:01.820601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:01.830944 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edee30d-b912-42c2-ae0b-ad39eaaf0579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:01.831778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edee30d-b912-42c2-ae0b-ad39eaaf0579 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:01.832207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edee30d-b912-42c2-ae0b-ad39eaaf0579 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:01.832589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edee30d-b912-42c2-ae0b-ad39eaaf0579 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:01.833547 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edee30d-b912-42c2-ae0b-ad39eaaf0579 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:01.833959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edee30d-b912-42c2-ae0b-ad39eaaf0579 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:01.835594 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edee30d-b912-42c2-ae0b-ad39eaaf0579 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:01.849496 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 477/934] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Jul 19 15:18:01 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:01.859648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bce58-0f12-4864-a5a2-ed2d8e99a5e5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:01.900366 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f7cc0-9b2a-431b-90c1-5d17a2bc1e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:01.901360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f7cc0-9b2a-431b-90c1-5d17a2bc1e7b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:01.901794 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f7cc0-9b2a-431b-90c1-5d17a2bc1e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:01.902216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f7cc0-9b2a-431b-90c1-5d17a2bc1e7b None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:01.903153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f7cc0-9b2a-431b-90c1-5d17a2bc1e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:01.903626 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f7cc0-9b2a-431b-90c1-5d17a2bc1e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:01.905237 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f7cc0-9b2a-431b-90c1-5d17a2bc1e7b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:01.924023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 458/935] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:01.929553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edee30d-b912-42c2-ae0b-ad39eaaf0579 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:02.043602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edee30d-b912-42c2-ae0b-ad39eaaf0579 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:02.078915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8820e7-c9fe-45e9-9ae8-54e6528aad6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:02.079742 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8820e7-c9fe-45e9-9ae8-54e6528aad6f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:02.080019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8820e7-c9fe-45e9-9ae8-54e6528aad6f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:02.080290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8820e7-c9fe-45e9-9ae8-54e6528aad6f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:02.088190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8820e7-c9fe-45e9-9ae8-54e6528aad6f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:02.089663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8820e7-c9fe-45e9-9ae8-54e6528aad6f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-1f8820e7-c9fe-45e9-9ae8-54e6528aad6f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:18:02.104716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:02.105760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:02.106775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:02.106775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:02.106775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:18:02.106775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:18:02.106775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:18:02.106775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:18:02.106775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:18:02.106775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:18:02.106775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 15:18:02.106775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:02.106775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 478/936] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Jul 19 15:18:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:02.114446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f7cc0-9b2a-431b-90c1-5d17a2bc1e7b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:02.223931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f7cc0-9b2a-431b-90c1-5d17a2bc1e7b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:02.238417 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:02.260098 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78baa34b-19db-4833-92e5-29eb90ebeb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:02.260790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78baa34b-19db-4833-92e5-29eb90ebeb81 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:02.261156 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78baa34b-19db-4833-92e5-29eb90ebeb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:02.261660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78baa34b-19db-4833-92e5-29eb90ebeb81 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:02.262887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78baa34b-19db-4833-92e5-29eb90ebeb81 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:02.263198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78baa34b-19db-4833-92e5-29eb90ebeb81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:02.264693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78baa34b-19db-4833-92e5-29eb90ebeb81 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:02.284444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 459/937] 10.222.0.42 () {56 vars in 1149 bytes} [Tue Jul 19 15:18:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:02.289783 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8820e7-c9fe-45e9-9ae8-54e6528aad6f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:02.346069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:02.396123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8820e7-c9fe-45e9-9ae8-54e6528aad6f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:02.431203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c0323-cf41-4d87-9d8e-b25bc98a6889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:02.431998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c0323-cf41-4d87-9d8e-b25bc98a6889 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:02.432275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c0323-cf41-4d87-9d8e-b25bc98a6889 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:02.432546 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c0323-cf41-4d87-9d8e-b25bc98a6889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:02.444428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c0323-cf41-4d87-9d8e-b25bc98a6889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:02.446000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c0323-cf41-4d87-9d8e-b25bc98a6889 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:02.467664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-f14c0323-cf41-4d87-9d8e-b25bc98a6889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:02.467664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:02.467664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:02.467664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:02.467664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:02.467664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:02.467664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:02.467664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:02.467664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:02.467664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:02.467664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:02.467664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:02.467664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:18:02.467664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:18:02.467664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:02.468630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:02.470009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:18:02.470009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:18:02.470009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:02.470009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:02.470009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 479/938] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:02.478393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78baa34b-19db-4833-92e5-29eb90ebeb81 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:02.591083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78baa34b-19db-4833-92e5-29eb90ebeb81 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:02.628027 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a26e44-7320-4ec2-b36a-dfa1024ed616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:02.628613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a26e44-7320-4ec2-b36a-dfa1024ed616 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:02.628892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a26e44-7320-4ec2-b36a-dfa1024ed616 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:02.629185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a26e44-7320-4ec2-b36a-dfa1024ed616 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:02.629918 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a26e44-7320-4ec2-b36a-dfa1024ed616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:02.630224 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a26e44-7320-4ec2-b36a-dfa1024ed616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:02.631591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a26e44-7320-4ec2-b36a-dfa1024ed616 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:02.644845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 460/939] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:02.650560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c0323-cf41-4d87-9d8e-b25bc98a6889 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:02.673043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1a3fd-a736-4c23-8c90-7c1a54ff9f74 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:02.673410 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1a3fd-a736-4c23-8c90-7c1a54ff9f74 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:02.673813 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1a3fd-a736-4c23-8c90-7c1a54ff9f74 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:02.675672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 461/940] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:02.680689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859c585-6374-4a6f-ab5a-f94487423399 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:02.680974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859c585-6374-4a6f-ab5a-f94487423399 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:02.681278 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859c585-6374-4a6f-ab5a-f94487423399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:02.699365 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3859c585-6374-4a6f-ab5a-f94487423399 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:02.700697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3859c585-6374-4a6f-ab5a-f94487423399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:02.715743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:02.765882 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c0323-cf41-4d87-9d8e-b25bc98a6889 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:02.769155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:02.777482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 462/941] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:02.800841 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859c585-6374-4a6f-ab5a-f94487423399 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:02.807143 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f726ca-2797-402e-a5ed-875cf3310d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:02.808197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f726ca-2797-402e-a5ed-875cf3310d2c None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:02.808729 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f726ca-2797-402e-a5ed-875cf3310d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:02.809240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f726ca-2797-402e-a5ed-875cf3310d2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:02.817492 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f726ca-2797-402e-a5ed-875cf3310d2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:02.819247 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f726ca-2797-402e-a5ed-875cf3310d2c None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:02.836242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-60f726ca-2797-402e-a5ed-875cf3310d2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:18:02.836242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:02.836242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:02.836242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:02.836242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:02.836242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:02.836242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:02.836242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:02.836242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:02.836242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:02.836242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:02.836242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:02.836242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:18:02.836242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:18:02.836242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:02.837326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:02.838228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:18:02.838228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:18:02.838228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:18:02.838228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:18:02.838228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:18:02.838228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:18:02.838228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:18:02.838228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:02.841171 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 480/942] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:18:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:02.850911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f726ca-2797-402e-a5ed-875cf3310d2c None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:02.906510 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859c585-6374-4a6f-ab5a-f94487423399 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:02.941603 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd61dd7-1972-4826-b97a-9bc6f84bc085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:02.942386 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd61dd7-1972-4826-b97a-9bc6f84bc085 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:02.942674 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd61dd7-1972-4826-b97a-9bc6f84bc085 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:02.942958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd61dd7-1972-4826-b97a-9bc6f84bc085 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:02.943680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd61dd7-1972-4826-b97a-9bc6f84bc085 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:02.945112 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd61dd7-1972-4826-b97a-9bc6f84bc085 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:02.946507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: WARNING keystone.server.flask.common [None req-9bd61dd7-1972-4826-b97a-9bc6f84bc085 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 19 15:18:02.963466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f726ca-2797-402e-a5ed-875cf3310d2c None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:02.963941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.sql.core [None req-9bd61dd7-1972-4826-b97a-9bc6f84bc085 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 19 15:18:02.963941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 19 15:18:02.963941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [parameters: {'id': 'c0680e2db9ba4c7fad79ccb5142059ac', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 19 15:18:02.963941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80270) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 19 15:18:02.971898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9bd61dd7-1972-4826-b97a-9bc6f84bc085 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c0680e2db9ba4c7fad79ccb5142059ac.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c0680e2db9ba4c7fad79ccb5142059ac. Jul 19 15:18:02.971898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:02.971898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 15:18:02.971898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 15:18:02.971898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 15:18:02.971898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 15:18:02.971898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 15:18:02.971898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 15:18:02.971898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 15:18:02.971898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application conn.query(q) Jul 19 15:18:02.971898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 15:18:02.971898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 15:18:02.971898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result.read() Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 19 15:18:02.972740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application next(self.gen) Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application yield resource Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application next(self.gen) Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.session.rollback() Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.force_reraise() Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise self.value Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application session.commit() Jul 19 15:18:02.973745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._prepare_impl() Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.session.flush() Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._flush(objects) Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application compat.raise_( Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application flush_context.execute() Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 19 15:18:02.974655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rec.execute(self) Jul 19 15:18:02.975599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 19 15:18:02.975599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 19 15:18:02.975599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 19 15:18:02.975599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 19 15:18:02.975599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 19 15:18:02.975599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 19 15:18:02.975599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 19 15:18:02.975599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 19 15:18:02.975599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 19 15:18:02.975599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 19 15:18:02.975599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 19 15:18:02.975599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 19 15:18:02.975599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 19 15:18:02.975599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 19 15:18:02.975599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application conn.query(q) Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result.read() Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 15:18:02.976397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 15:18:02.978168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 15:18:02.978168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 15:18:02.978168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 15:18:02.978168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 15:18:02.978168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 15:18:02.978168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 15:18:02.978168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 19 15:18:02.978168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 19 15:18:02.978168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [parameters: {'id': 'c0680e2db9ba4c7fad79ccb5142059ac', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 19 15:18:02.978168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 19 15:18:02.978168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:02.978168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:18:02.978168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:02.978168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:02.978168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:02.979680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:02.981657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:02.981657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 19 15:18:02.981657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 19 15:18:02.981657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:02.981657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:02.981657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 19 15:18:02.981657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 19 15:18:02.981657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c0680e2db9ba4c7fad79ccb5142059ac. Jul 19 15:18:02.981657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:02.981657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 463/943] 10.222.0.42 () {60 vars in 1140 bytes} [Tue Jul 19 15:18:02 2022] POST /identity/v3/projects => generated 262 bytes in 173 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:02.981657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fea46-74e4-45c8-9ea3-3b2d37e5427a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:02.981657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fea46-74e4-45c8-9ea3-3b2d37e5427a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:02.983221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fea46-74e4-45c8-9ea3-3b2d37e5427a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:02.983221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 464/944] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:02.984100 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fea46-74e4-45c8-9ea3-3b2d37e5427a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:02.993761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:03.016813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a1a7d-2335-45b7-b849-de210e61c34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:03.017588 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a1a7d-2335-45b7-b849-de210e61c34e None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:03.017871 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a1a7d-2335-45b7-b849-de210e61c34e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:03.018157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a1a7d-2335-45b7-b849-de210e61c34e None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:03.018961 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a1a7d-2335-45b7-b849-de210e61c34e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:03.019274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a1a7d-2335-45b7-b849-de210e61c34e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:03.020743 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a1a7d-2335-45b7-b849-de210e61c34e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:03.034947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 481/945] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:18:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:03.038910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa972c-f22c-4779-8896-d1f1a0a7dd04 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:03.039470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa972c-f22c-4779-8896-d1f1a0a7dd04 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:03.039991 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa972c-f22c-4779-8896-d1f1a0a7dd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:03.057660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-2faa972c-f22c-4779-8896-d1f1a0a7dd04 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:03.058828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2faa972c-f22c-4779-8896-d1f1a0a7dd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:03.081631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:03.098749 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fea46-74e4-45c8-9ea3-3b2d37e5427a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:03.136415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba2a03-c02b-46d3-bf84-1db24a0605a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:03.137249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba2a03-c02b-46d3-bf84-1db24a0605a9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:03.137540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba2a03-c02b-46d3-bf84-1db24a0605a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:03.137825 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba2a03-c02b-46d3-bf84-1db24a0605a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:03.139566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 482/946] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:03.142264 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa972c-f22c-4779-8896-d1f1a0a7dd04 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:03.145903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba2a03-c02b-46d3-bf84-1db24a0605a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:03.147723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba2a03-c02b-46d3-bf84-1db24a0605a9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:03.165371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d7ba2a03-c02b-46d3-bf84-1db24a0605a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:18:03.165371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:03.165371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:03.165371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:03.165371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:03.165371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:03.165371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:03.165371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:03.165371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:03.165371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:03.165371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:03.165371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:03.165371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:18:03.165371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:18:03.165371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:03.166333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:03.167251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:18:03.167251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:18:03.167251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:18:03.167251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:18:03.167251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:18:03.167251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:18:03.167251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:18:03.167251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:03.171163 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 465/947] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:18:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:03.171915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba2a03-c02b-46d3-bf84-1db24a0605a9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:03.243332 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:03.261094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa972c-f22c-4779-8896-d1f1a0a7dd04 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:03.285088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba2a03-c02b-46d3-bf84-1db24a0605a9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:03.295533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75568e5-6ded-4ff9-bb89-f4456e8d5de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:03.296420 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75568e5-6ded-4ff9-bb89-f4456e8d5de2 None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:03.296707 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75568e5-6ded-4ff9-bb89-f4456e8d5de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:03.296985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75568e5-6ded-4ff9-bb89-f4456e8d5de2 None admin] PATH_INFO: `/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/60f89f6042664f91b2829d516ed7e844/roles/d7b232d8f46040ba8eab9538a784e61f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:03.300152 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75568e5-6ded-4ff9-bb89-f4456e8d5de2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5f51e47b8e34bf5ac822d8c93853e30, user_id=60f89f6042664f91b2829d516ed7e844, role_id=d7b232d8f46040ba8eab9538a784e61f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:03.301630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75568e5-6ded-4ff9-bb89-f4456e8d5de2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:03.314448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 483/948] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:18:03 2022] PUT /identity/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/60f89f6042664f91b2829d516ed7e844/roles/d7b232d8f46040ba8eab9538a784e61f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:18:03.315234 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75568e5-6ded-4ff9-bb89-f4456e8d5de2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:03.333963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da9d3f-86a0-4e8f-81cd-856a415160ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:03.335265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da9d3f-86a0-4e8f-81cd-856a415160ca None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:03.335782 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da9d3f-86a0-4e8f-81cd-856a415160ca None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:03.336409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da9d3f-86a0-4e8f-81cd-856a415160ca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:03.337390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da9d3f-86a0-4e8f-81cd-856a415160ca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:03.339190 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da9d3f-86a0-4e8f-81cd-856a415160ca None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:03.339760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-50da9d3f-86a0-4e8f-81cd-856a415160ca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:03.355697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:03.356894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:03.357884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:03.357884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:03.357884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:03.357884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:03.357884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:03.357884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:03.357884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:03.357884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:03.357884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:03.357884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:03.357884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:18:03.357884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:03.357884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 15:18:03.357884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:03.361752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 466/949] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Jul 19 15:18:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:03.368875 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da9d3f-86a0-4e8f-81cd-856a415160ca None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:03.468249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75568e5-6ded-4ff9-bb89-f4456e8d5de2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:03.486243 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da9d3f-86a0-4e8f-81cd-856a415160ca None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:03.517956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202ee50-2eb7-408d-a6f9-261110466cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:03.518836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202ee50-2eb7-408d-a6f9-261110466cbd None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:03.519669 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202ee50-2eb7-408d-a6f9-261110466cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:03.520055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202ee50-2eb7-408d-a6f9-261110466cbd None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:03.521253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202ee50-2eb7-408d-a6f9-261110466cbd None admin] RBAC: Adding query filter params (name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:03.521671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202ee50-2eb7-408d-a6f9-261110466cbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:03.523682 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202ee50-2eb7-408d-a6f9-261110466cbd None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:03.528078 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:03.534133 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8ba1c-66ac-423e-8845-624a4ff3377e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:03.535278 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8ba1c-66ac-423e-8845-624a4ff3377e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:03.535950 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8ba1c-66ac-423e-8845-624a4ff3377e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:03.536234 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8ba1c-66ac-423e-8845-624a4ff3377e None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:03.536962 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8ba1c-66ac-423e-8845-624a4ff3377e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:03.537323 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8ba1c-66ac-423e-8845-624a4ff3377e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:03.538701 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8ba1c-66ac-423e-8845-624a4ff3377e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:03.543699 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 484/950] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Jul 19 15:18:03 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:03.551080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 467/951] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Jul 19 15:18:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:03.557282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202ee50-2eb7-408d-a6f9-261110466cbd None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:03.643535 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:03.677818 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202ee50-2eb7-408d-a6f9-261110466cbd None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:03.717246 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84998537-e69e-4ac2-836b-838efb21daa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:03.718354 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84998537-e69e-4ac2-836b-838efb21daa0 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:03.718800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84998537-e69e-4ac2-836b-838efb21daa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:03.719079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84998537-e69e-4ac2-836b-838efb21daa0 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:03.720451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84998537-e69e-4ac2-836b-838efb21daa0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:03.720864 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84998537-e69e-4ac2-836b-838efb21daa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:03.722386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84998537-e69e-4ac2-836b-838efb21daa0 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:03.728388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 485/952] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Jul 19 15:18:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:03.734607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8ba1c-66ac-423e-8845-624a4ff3377e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:03.844263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8ba1c-66ac-423e-8845-624a4ff3377e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:03.880096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e37a3-80f6-42b7-9ab4-20715ee0b7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:03.880891 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e37a3-80f6-42b7-9ab4-20715ee0b7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:03.881198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e37a3-80f6-42b7-9ab4-20715ee0b7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:03.881476 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e37a3-80f6-42b7-9ab4-20715ee0b7f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:03.882282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e37a3-80f6-42b7-9ab4-20715ee0b7f5 None admin] RBAC: Adding query filter params (interface=public, service_id=3bc9cf4cf5674d4c9b552aaa22e74e07, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:03.882589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e37a3-80f6-42b7-9ab4-20715ee0b7f5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:03.883949 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e37a3-80f6-42b7-9ab4-20715ee0b7f5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:03.889793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 468/953] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Jul 19 15:18:03 2022] GET /identity/v3/endpoints?service_id=3bc9cf4cf5674d4c9b552aaa22e74e07&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:03.896018 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84998537-e69e-4ac2-836b-838efb21daa0 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:04.010910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84998537-e69e-4ac2-836b-838efb21daa0 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:04.033593 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754035c-6e75-4dae-8638-13e5433f080b None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:04.033902 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754035c-6e75-4dae-8638-13e5433f080b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:04.034159 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754035c-6e75-4dae-8638-13e5433f080b None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:04.035784 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 469/954] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:04.040861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a01d4f-fcba-4c7c-bea4-0851ae0b24ba None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:04.041162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a01d4f-fcba-4c7c-bea4-0851ae0b24ba None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:04.041461 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a01d4f-fcba-4c7c-bea4-0851ae0b24ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:04.048803 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0a775-f84a-425b-8087-c39960ce53eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:04.049503 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0a775-f84a-425b-8087-c39960ce53eb None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:04.049793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0a775-f84a-425b-8087-c39960ce53eb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:04.050069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0a775-f84a-425b-8087-c39960ce53eb None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:04.050843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0a775-f84a-425b-8087-c39960ce53eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:04.052285 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0a775-f84a-425b-8087-c39960ce53eb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:04.058902 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 486/955] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Jul 19 15:18:03 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:18:04.062529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-61a01d4f-fcba-4c7c-bea4-0851ae0b24ba None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:04.064272 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-61a01d4f-fcba-4c7c-bea4-0851ae0b24ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:04.130159 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:04.147073 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 470/956] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:04.170147 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0a775-f84a-425b-8087-c39960ce53eb None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:04.174337 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:04.286666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0a775-f84a-425b-8087-c39960ce53eb None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:04.322170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5bf3e-70e1-435b-bc7b-89014ccf5382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:04.322890 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5bf3e-70e1-435b-bc7b-89014ccf5382 None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:04.323181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5bf3e-70e1-435b-bc7b-89014ccf5382 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:04.323462 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5bf3e-70e1-435b-bc7b-89014ccf5382 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:04.324206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5bf3e-70e1-435b-bc7b-89014ccf5382 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:04.325630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5bf3e-70e1-435b-bc7b-89014ccf5382 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:04.342590 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 487/957] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Jul 19 15:18:04 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:18:05.450460 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796952d1-53f4-4c22-b41e-20984acd34e0 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:05.451009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796952d1-53f4-4c22-b41e-20984acd34e0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:05.451009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796952d1-53f4-4c22-b41e-20984acd34e0 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:05.452451 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 471/958] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:05.459017 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cf06c-8dd6-4a32-8547-32cfef0e2f11 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:05.459516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cf06c-8dd6-4a32-8547-32cfef0e2f11 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:05.459621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cf06c-8dd6-4a32-8547-32cfef0e2f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:05.481193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-3a6cf06c-8dd6-4a32-8547-32cfef0e2f11 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:05.482387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3a6cf06c-8dd6-4a32-8547-32cfef0e2f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:05.542007 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:05.565247 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 488/959] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:05.585520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796952d1-53f4-4c22-b41e-20984acd34e0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:05.691369 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796952d1-53f4-4c22-b41e-20984acd34e0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:05.732329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438ed62-b6dd-441f-bcd5-e4f214918d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:05.733174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438ed62-b6dd-441f-bcd5-e4f214918d45 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:05.733455 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438ed62-b6dd-441f-bcd5-e4f214918d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:05.733731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438ed62-b6dd-441f-bcd5-e4f214918d45 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:05.745262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438ed62-b6dd-441f-bcd5-e4f214918d45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-8438ed62-b6dd-441f-bcd5-e4f214918d45 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:18:05.754271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:05.755263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:05.756150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:05.756150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:05.756150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:05.756150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:05.756150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:18:05.756150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:18:05.756150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:18:05.756150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:18:05.756150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:18:05.756150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:18:05.756150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 15:18:05.756150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:05.760387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 472/960] 10.222.0.42 () {56 vars in 1120 bytes} [Tue Jul 19 15:18:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:05.761752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cf06c-8dd6-4a32-8547-32cfef0e2f11 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:05.775448 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd522c-9e9a-4204-a20d-4ac1e87ee636 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:05.775739 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd522c-9e9a-4204-a20d-4ac1e87ee636 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:05.776025 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd522c-9e9a-4204-a20d-4ac1e87ee636 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:05.777705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 473/961] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:18:05.783187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14e16d-6479-4df9-9f13-5370bc657a21 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:05.783484 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14e16d-6479-4df9-9f13-5370bc657a21 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:05.783767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14e16d-6479-4df9-9f13-5370bc657a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:05.801811 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6c14e16d-6479-4df9-9f13-5370bc657a21 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:05.802935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6c14e16d-6479-4df9-9f13-5370bc657a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:05.864609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:05.876353 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cf06c-8dd6-4a32-8547-32cfef0e2f11 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:05.898177 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 474/962] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:18:05.922660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58a7a1-d226-4e80-a9ac-2062fc1c5eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:05.923686 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58a7a1-d226-4e80-a9ac-2062fc1c5eac None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:05.924342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58a7a1-d226-4e80-a9ac-2062fc1c5eac None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:05.924832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58a7a1-d226-4e80-a9ac-2062fc1c5eac None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:05.925178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14e16d-6479-4df9-9f13-5370bc657a21 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:05.926077 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58a7a1-d226-4e80-a9ac-2062fc1c5eac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:05.926601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58a7a1-d226-4e80-a9ac-2062fc1c5eac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:05.929265 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58a7a1-d226-4e80-a9ac-2062fc1c5eac None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:05.945628 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 489/963] 10.222.0.42 () {56 vars in 1115 bytes} [Tue Jul 19 15:18:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:05.947152 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58a7a1-d226-4e80-a9ac-2062fc1c5eac None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:06.001268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:06.034376 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14e16d-6479-4df9-9f13-5370bc657a21 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:06.055398 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58a7a1-d226-4e80-a9ac-2062fc1c5eac None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:06.073393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6ebab-f2b5-42af-8b2f-d7b78a39839e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:06.074086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6ebab-f2b5-42af-8b2f-d7b78a39839e None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:06.074373 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6ebab-f2b5-42af-8b2f-d7b78a39839e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:06.074655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6ebab-f2b5-42af-8b2f-d7b78a39839e None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:06.075786 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6ebab-f2b5-42af-8b2f-d7b78a39839e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:06.077256 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6ebab-f2b5-42af-8b2f-d7b78a39839e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:06.094741 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615b7e1-e564-45a3-b595-bc83badf1e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:06.095695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.sql.core [None req-0ac6ebab-f2b5-42af-8b2f-d7b78a39839e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 19 15:18:06.095695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 19 15:18:06.095695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [parameters: {'user_id': 'e5b9e19fcdab4337aadd1ca5d3b1d86a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 19 15:18:06.095695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80270) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 19 15:18:06.096399 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615b7e1-e564-45a3-b595-bc83badf1e73 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:06.096900 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615b7e1-e564-45a3-b595-bc83badf1e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:06.097414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615b7e1-e564-45a3-b595-bc83badf1e73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:06.104377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-0ac6ebab-f2b5-42af-8b2f-d7b78a39839e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 19 15:18:06.104377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:06.104377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 15:18:06.104377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 15:18:06.104377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 15:18:06.104377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 15:18:06.104377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 15:18:06.104377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 15:18:06.104377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 15:18:06.104377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application conn.query(q) Jul 19 15:18:06.104377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 15:18:06.104377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 15:18:06.104377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 15:18:06.104377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result.read() Jul 19 15:18:06.104377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 15:18:06.105295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application next(self.gen) Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application yield resource Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application next(self.gen) Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.session.rollback() Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.force_reraise() Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise self.value Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application session.commit() Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 19 15:18:06.106264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._prepare_impl() Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.session.flush() Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._flush(objects) Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application compat.raise_( Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application flush_context.execute() Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rec.execute(self) Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 19 15:18:06.107721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 15:18:06.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application conn.query(q) Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result.read() Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 15:18:06.109514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 15:18:06.112449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 15:18:06.112449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 15:18:06.112449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 15:18:06.112449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 15:18:06.112449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 19 15:18:06.112449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 19 15:18:06.112449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e5b9e19fcdab4337aadd1ca5d3b1d86a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 19 15:18:06.112449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 19 15:18:06.112449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:06.112449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:18:06.112449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:06.112449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:06.112449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:06.112449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:06.112449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 19 15:18:06.114346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 19 15:18:06.116415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 19 15:18:06.116415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 19 15:18:06.116415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 19 15:18:06.116415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 19 15:18:06.116415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 19 15:18:06.116415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:06.116415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 475/964] 10.222.0.42 () {60 vars in 1131 bytes} [Tue Jul 19 15:18:05 2022] POST /identity/v3/users => generated 160 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:18:06.116415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6ebab-f2b5-42af-8b2f-d7b78a39839e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:06.122123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615b7e1-e564-45a3-b595-bc83badf1e73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:06.123801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615b7e1-e564-45a3-b595-bc83badf1e73 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:06.142361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-a615b7e1-e564-45a3-b595-bc83badf1e73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:06.142361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:06.142361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:06.142361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:06.142361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:06.142361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:06.142361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:06.142361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:06.142361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:06.142361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:06.142361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:06.142361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:06.142361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:18:06.142361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:18:06.142361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:06.143308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:06.144203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:18:06.144203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:18:06.144203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:06.144203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:06.151197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 490/965] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:18:06.152734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615b7e1-e564-45a3-b595-bc83badf1e73 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:06.168231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:06.244504 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6ebab-f2b5-42af-8b2f-d7b78a39839e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:06.268874 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615b7e1-e564-45a3-b595-bc83badf1e73 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:06.270553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:06.285303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b034f-d4a4-4687-8fa3-2d7c8fc950b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:06.286388 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b034f-d4a4-4687-8fa3-2d7c8fc950b0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:06.286858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b034f-d4a4-4687-8fa3-2d7c8fc950b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:06.287141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b034f-d4a4-4687-8fa3-2d7c8fc950b0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:06.295464 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b034f-d4a4-4687-8fa3-2d7c8fc950b0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:06.297061 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b034f-d4a4-4687-8fa3-2d7c8fc950b0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:06.309802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb569c8a-613c-4ad7-95d3-b7863a5f238a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:06.310403 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb569c8a-613c-4ad7-95d3-b7863a5f238a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:06.310696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb569c8a-613c-4ad7-95d3-b7863a5f238a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:06.311105 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb569c8a-613c-4ad7-95d3-b7863a5f238a None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:06.311846 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb569c8a-613c-4ad7-95d3-b7863a5f238a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-259b034f-d4a4-4687-8fa3-2d7c8fc950b0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:18:06.312087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:06.313200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:06.314789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:06.314789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:06.314789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:18:06.314789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:18:06.314789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:18:06.314789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:18:06.314789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:18:06.314789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:18:06.314789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 15:18:06.314789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:06.315359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb569c8a-613c-4ad7-95d3-b7863a5f238a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:06.315359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb569c8a-613c-4ad7-95d3-b7863a5f238a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:06.318179 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 476/966] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:18:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:06.319612 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5399d-125c-4ac2-a2c9-c185704f2038 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:06.319905 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5399d-125c-4ac2-a2c9-c185704f2038 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:06.320186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5399d-125c-4ac2-a2c9-c185704f2038 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:06.329180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 477/967] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:06.329180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5399d-125c-4ac2-a2c9-c185704f2038 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:06.329670 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 491/968] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:06.330518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c1d3d-7e3d-4409-bf0e-3cbdcbbd8a73 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:06.330865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c1d3d-7e3d-4409-bf0e-3cbdcbbd8a73 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:06.331154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c1d3d-7e3d-4409-bf0e-3cbdcbbd8a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:06.349862 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-4b4c1d3d-7e3d-4409-bf0e-3cbdcbbd8a73 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:06.351057 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-4b4c1d3d-7e3d-4409-bf0e-3cbdcbbd8a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:06.429473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 492/969] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:06.429473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c1d3d-7e3d-4409-bf0e-3cbdcbbd8a73 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:06.448867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:06.456797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5399d-125c-4ac2-a2c9-c185704f2038 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:06.494096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eca425-dc8d-415f-90ad-53a0122bd4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:06.494782 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eca425-dc8d-415f-90ad-53a0122bd4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:06.495069 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eca425-dc8d-415f-90ad-53a0122bd4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:06.495350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eca425-dc8d-415f-90ad-53a0122bd4e6 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:06.496163 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eca425-dc8d-415f-90ad-53a0122bd4e6 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:06.496474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eca425-dc8d-415f-90ad-53a0122bd4e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:06.497930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eca425-dc8d-415f-90ad-53a0122bd4e6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:06.534843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 478/970] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Jul 19 15:18:06 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:06.535695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eca425-dc8d-415f-90ad-53a0122bd4e6 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:06.545113 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:06.562623 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c1d3d-7e3d-4409-bf0e-3cbdcbbd8a73 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:06.614157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd799c22-940d-4d30-a3fc-9016cd82b24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:06.615149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd799c22-940d-4d30-a3fc-9016cd82b24f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:06.615493 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd799c22-940d-4d30-a3fc-9016cd82b24f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:06.615998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd799c22-940d-4d30-a3fc-9016cd82b24f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:06.626586 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd799c22-940d-4d30-a3fc-9016cd82b24f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:06.629052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd799c22-940d-4d30-a3fc-9016cd82b24f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-fd799c22-940d-4d30-a3fc-9016cd82b24f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:18:06.647675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:06.648870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:06.650409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:06.650409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:06.650409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:18:06.650409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:18:06.650409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:18:06.650409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:18:06.650409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:18:06.650409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:18:06.650409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 15:18:06.650409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:06.650409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 493/971] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Jul 19 15:18:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:06.651485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8f928-2a0b-453c-89f7-41376bea1dff None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:06.651610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8f928-2a0b-453c-89f7-41376bea1dff None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:06.651870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8f928-2a0b-453c-89f7-41376bea1dff None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:06.653409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 494/972] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:06.655811 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8f928-2a0b-453c-89f7-41376bea1dff None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:06.695025 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eca425-dc8d-415f-90ad-53a0122bd4e6 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:06.740897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ce0fc-3083-437a-8788-90fb5a6563e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:06.741749 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ce0fc-3083-437a-8788-90fb5a6563e6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:06.743074 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ce0fc-3083-437a-8788-90fb5a6563e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:06.743074 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ce0fc-3083-437a-8788-90fb5a6563e6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:06.743074 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ce0fc-3083-437a-8788-90fb5a6563e6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:06.744340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ce0fc-3083-437a-8788-90fb5a6563e6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-683ce0fc-3083-437a-8788-90fb5a6563e6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:06.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:06.760515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:06.761821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:06.761821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:06.761821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:06.761821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:06.761821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:06.761821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:06.761821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:06.761821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:06.761821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:06.761821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:06.761821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:18:06.761821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:06.761821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 15:18:06.761821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:06.761821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 479/973] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Jul 19 15:18:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:06.762683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c8073-168c-475d-8516-913088839475 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:06.762960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c8073-168c-475d-8516-913088839475 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:06.763612 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c8073-168c-475d-8516-913088839475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:06.776485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8f928-2a0b-453c-89f7-41376bea1dff None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:06.782583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-da3c8073-168c-475d-8516-913088839475 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:06.784135 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-da3c8073-168c-475d-8516-913088839475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:06.806209 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:06.822690 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5300337-8477-4405-acee-5fe35b329a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:06.823293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5300337-8477-4405-acee-5fe35b329a67 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:06.823579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5300337-8477-4405-acee-5fe35b329a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:06.823859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5300337-8477-4405-acee-5fe35b329a67 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:06.824764 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5300337-8477-4405-acee-5fe35b329a67 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:06.825019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5300337-8477-4405-acee-5fe35b329a67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:06.827109 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5300337-8477-4405-acee-5fe35b329a67 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:06.844529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 495/974] 10.222.0.42 () {56 vars in 1149 bytes} [Tue Jul 19 15:18:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:06.845352 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5300337-8477-4405-acee-5fe35b329a67 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:06.857222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:06.861945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 480/975] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:06.862775 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c8073-168c-475d-8516-913088839475 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:06.958378 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5300337-8477-4405-acee-5fe35b329a67 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:06.962420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c8073-168c-475d-8516-913088839475 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:06.992264 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00f2ce-b69d-44ee-b382-3a5586b2b60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:06.993007 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00f2ce-b69d-44ee-b382-3a5586b2b60e None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:06.993377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00f2ce-b69d-44ee-b382-3a5586b2b60e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:06.993658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00f2ce-b69d-44ee-b382-3a5586b2b60e None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:06.994372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00f2ce-b69d-44ee-b382-3a5586b2b60e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:06.994683 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00f2ce-b69d-44ee-b382-3a5586b2b60e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:06.996053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00f2ce-b69d-44ee-b382-3a5586b2b60e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:06.996793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4b969-22d4-4a06-96b4-adbde78f66f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:06.997890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4b969-22d4-4a06-96b4-adbde78f66f8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:06.998192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4b969-22d4-4a06-96b4-adbde78f66f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:06.998466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4b969-22d4-4a06-96b4-adbde78f66f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:07.000887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 496/976] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Jul 19 15:18:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:07.001687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00f2ce-b69d-44ee-b382-3a5586b2b60e None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:07.020582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:07.066764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4b969-22d4-4a06-96b4-adbde78f66f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:07.068214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4b969-22d4-4a06-96b4-adbde78f66f8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:07.076205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:07.103446 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00f2ce-b69d-44ee-b382-3a5586b2b60e None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:07.138005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ef747-311a-47cb-bb56-3b03712fe1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:07.138797 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ef747-311a-47cb-bb56-3b03712fe1fc None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:07.139081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ef747-311a-47cb-bb56-3b03712fe1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:07.139357 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ef747-311a-47cb-bb56-3b03712fe1fc None admin] PATH_INFO: `/v3/services/compute` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:07.140010 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ef747-311a-47cb-bb56-3b03712fe1fc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:07.141944 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ef747-311a-47cb-bb56-3b03712fe1fc None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-b71ef747-311a-47cb-bb56-3b03712fe1fc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:07.154069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:07.155240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:07.156130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:07.156130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:07.156130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:07.156130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:07.156130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:07.156130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:07.156130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:07.156130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:07.156130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:07.156130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:07.156130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:18:07.156130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:07.156130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 15:18:07.156130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:07.157464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 497/977] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:18:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:07.158195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ef747-311a-47cb-bb56-3b03712fe1fc None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:07.169094 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 481/978] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:18:06 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:18:07.169948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4b969-22d4-4a06-96b4-adbde78f66f8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:07.222628 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:07.260810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ef747-311a-47cb-bb56-3b03712fe1fc None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:07.271180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4b969-22d4-4a06-96b4-adbde78f66f8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:07.294732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e235e7a-a5f2-4d6b-b908-a635e0310ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:07.295400 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e235e7a-a5f2-4d6b-b908-a635e0310ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:07.295689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e235e7a-a5f2-4d6b-b908-a635e0310ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:07.295971 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e235e7a-a5f2-4d6b-b908-a635e0310ea7 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:07.296709 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e235e7a-a5f2-4d6b-b908-a635e0310ea7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:07.297020 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e235e7a-a5f2-4d6b-b908-a635e0310ea7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:07.298464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e235e7a-a5f2-4d6b-b908-a635e0310ea7 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:07.303414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 498/979] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Jul 19 15:18:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:07.304003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e235e7a-a5f2-4d6b-b908-a635e0310ea7 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:07.306492 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:07.315884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36067f08-4282-4742-9804-d31903f89e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:07.316558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36067f08-4282-4742-9804-d31903f89e48 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:07.316844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36067f08-4282-4742-9804-d31903f89e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:07.317120 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36067f08-4282-4742-9804-d31903f89e48 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:07.317868 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36067f08-4282-4742-9804-d31903f89e48 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:07.318189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36067f08-4282-4742-9804-d31903f89e48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:07.319556 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36067f08-4282-4742-9804-d31903f89e48 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:07.325105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 482/980] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Jul 19 15:18:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:07.325747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36067f08-4282-4742-9804-d31903f89e48 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:07.405364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e235e7a-a5f2-4d6b-b908-a635e0310ea7 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:07.423767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36067f08-4282-4742-9804-d31903f89e48 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:07.438804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0913f-6e3a-4fae-a469-41fc0d20a794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:07.439589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0913f-6e3a-4fae-a469-41fc0d20a794 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:07.439877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0913f-6e3a-4fae-a469-41fc0d20a794 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:07.440151 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0913f-6e3a-4fae-a469-41fc0d20a794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:07.440817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:07.451114 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0913f-6e3a-4fae-a469-41fc0d20a794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:07.452561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0913f-6e3a-4fae-a469-41fc0d20a794 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:07.459879 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341c362-1d6a-4ee6-8db6-aee44101e657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:07.460526 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341c362-1d6a-4ee6-8db6-aee44101e657 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:07.460807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341c362-1d6a-4ee6-8db6-aee44101e657 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:07.461082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341c362-1d6a-4ee6-8db6-aee44101e657 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:07.461842 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341c362-1d6a-4ee6-8db6-aee44101e657 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:07.463184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341c362-1d6a-4ee6-8db6-aee44101e657 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:07.470283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-4bd0913f-6e3a-4fae-a469-41fc0d20a794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:07.470283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:07.470283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:07.470283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:07.470283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:07.470283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:07.470283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:07.470283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:07.470283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:07.470283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:07.470283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:07.470283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:07.470283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:18:07.470283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:18:07.470283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:07.471251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:07.472342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:18:07.472342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:18:07.472342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:07.472342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:07.472342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 499/981] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:07.472842 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0913f-6e3a-4fae-a469-41fc0d20a794 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:07.487648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 483/982] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Jul 19 15:18:07 2022] POST /identity/v3/endpoints => generated 349 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:18:07.488725 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341c362-1d6a-4ee6-8db6-aee44101e657 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:07.537788 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:07.591426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0913f-6e3a-4fae-a469-41fc0d20a794 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:07.620493 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341c362-1d6a-4ee6-8db6-aee44101e657 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:07.632719 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c902c-a6a0-4179-87a6-949fc931f5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:07.633794 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c902c-a6a0-4179-87a6-949fc931f5df None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:07.634192 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c902c-a6a0-4179-87a6-949fc931f5df None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:07.634470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c902c-a6a0-4179-87a6-949fc931f5df None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:07.635189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c902c-a6a0-4179-87a6-949fc931f5df None admin] RBAC: Adding query filter params (type=compute) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:07.635501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c902c-a6a0-4179-87a6-949fc931f5df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:07.636874 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c902c-a6a0-4179-87a6-949fc931f5df None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:07.642927 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 500/983] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Jul 19 15:18:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:07.648579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c902c-a6a0-4179-87a6-949fc931f5df None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:07.668175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e8dc0-0510-4acf-90d4-6b1a03752795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:07.669352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e8dc0-0510-4acf-90d4-6b1a03752795 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:07.670121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e8dc0-0510-4acf-90d4-6b1a03752795 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:07.670600 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e8dc0-0510-4acf-90d4-6b1a03752795 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:07.671573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e8dc0-0510-4acf-90d4-6b1a03752795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:07.672067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e8dc0-0510-4acf-90d4-6b1a03752795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:07.674057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e8dc0-0510-4acf-90d4-6b1a03752795 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:07.686937 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 484/984] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:07.692866 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e8dc0-0510-4acf-90d4-6b1a03752795 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:07.723722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:07.753766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c902c-a6a0-4179-87a6-949fc931f5df None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:07.770917 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:07.789094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17834db3-ec70-4537-86db-28b895098839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:07.789787 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17834db3-ec70-4537-86db-28b895098839 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:07.790080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17834db3-ec70-4537-86db-28b895098839 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:07.790357 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17834db3-ec70-4537-86db-28b895098839 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:07.791155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17834db3-ec70-4537-86db-28b895098839 None admin] RBAC: Adding query filter params (interface=public, service_id=7b56abf3cc614071b6774d348e5cf809, region_id=RegionOne) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:07.791466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17834db3-ec70-4537-86db-28b895098839 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:07.792838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17834db3-ec70-4537-86db-28b895098839 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:07.801833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 501/985] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Jul 19 15:18:07 2022] GET /identity/v3/endpoints?service_id=7b56abf3cc614071b6774d348e5cf809&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:07.803366 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e8dc0-0510-4acf-90d4-6b1a03752795 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:07.807271 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17834db3-ec70-4537-86db-28b895098839 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:07.842346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa28390-8f86-42ba-ba0d-d3fadbf76f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:07.843150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa28390-8f86-42ba-ba0d-d3fadbf76f01 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:07.843436 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa28390-8f86-42ba-ba0d-d3fadbf76f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:07.843719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa28390-8f86-42ba-ba0d-d3fadbf76f01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:07.852822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa28390-8f86-42ba-ba0d-d3fadbf76f01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:07.854721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa28390-8f86-42ba-ba0d-d3fadbf76f01 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:07.870189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-faa28390-8f86-42ba-ba0d-d3fadbf76f01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:18:07.870189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:07.870189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:07.870189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:07.870189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:07.870189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:07.870189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:07.870189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:07.870189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:07.870189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:07.870189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:07.870189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:07.870189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:18:07.870189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:18:07.870189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:07.871293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:07.872779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:18:07.872779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:18:07.872779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:18:07.872779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:18:07.872779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:18:07.872779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:18:07.872779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:18:07.872779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:07.872779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 485/986] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:18:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:18:07.877300 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa28390-8f86-42ba-ba0d-d3fadbf76f01 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:07.918448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17834db3-ec70-4537-86db-28b895098839 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:07.957717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5e30c-46fd-41e4-9b6e-0230eb62a512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:07.958377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5e30c-46fd-41e4-9b6e-0230eb62a512 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:07.958659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5e30c-46fd-41e4-9b6e-0230eb62a512 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:07.959233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5e30c-46fd-41e4-9b6e-0230eb62a512 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:07.959653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5e30c-46fd-41e4-9b6e-0230eb62a512 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:07.961063 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5e30c-46fd-41e4-9b6e-0230eb62a512 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:07.968008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 502/987] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Jul 19 15:18:07 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:18:07.976740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:07.992694 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa28390-8f86-42ba-ba0d-d3fadbf76f01 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:08.032216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169932c5-3d50-46e1-95cf-6444357fba87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:08.033152 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169932c5-3d50-46e1-95cf-6444357fba87 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:08.033498 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169932c5-3d50-46e1-95cf-6444357fba87 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:08.033718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169932c5-3d50-46e1-95cf-6444357fba87 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:08.034525 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169932c5-3d50-46e1-95cf-6444357fba87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:08.034838 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169932c5-3d50-46e1-95cf-6444357fba87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:08.036319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169932c5-3d50-46e1-95cf-6444357fba87 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:08.055297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 486/988] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:18:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:08.056626 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5e30c-46fd-41e4-9b6e-0230eb62a512 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:08.108242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:08.166678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5e30c-46fd-41e4-9b6e-0230eb62a512 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:08.201672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e440258-0a0d-446e-a6de-410ac714eb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:08.202252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e440258-0a0d-446e-a6de-410ac714eb64 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:08.202539 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e440258-0a0d-446e-a6de-410ac714eb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:08.202816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e440258-0a0d-446e-a6de-410ac714eb64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:08.204540 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e440258-0a0d-446e-a6de-410ac714eb64 None admin] RBAC: Adding query filter params (role.id=d7b232d8f46040ba8eab9538a784e61f, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=60f89f6042664f91b2829d516ed7e844) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:08.204936 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e440258-0a0d-446e-a6de-410ac714eb64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:08.206516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e440258-0a0d-446e-a6de-410ac714eb64 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:08.212844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 503/989] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:18:08 2022] GET /identity/v3/role_assignments?user.id=60f89f6042664f91b2829d516ed7e844&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=d7b232d8f46040ba8eab9538a784e61f => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:09.553693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373e86e-e2f2-426e-a124-b6d85f166624 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:09.554187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373e86e-e2f2-426e-a124-b6d85f166624 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:09.554187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373e86e-e2f2-426e-a124-b6d85f166624 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:09.556541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 487/990] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:09.561091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3db1f9-77a0-4fdd-9b9a-8161d366b9ef None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:09.561557 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3db1f9-77a0-4fdd-9b9a-8161d366b9ef None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:09.561738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3db1f9-77a0-4fdd-9b9a-8161d366b9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:09.579444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-0e3db1f9-77a0-4fdd-9b9a-8161d366b9ef None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:09.580732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0e3db1f9-77a0-4fdd-9b9a-8161d366b9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:09.650633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 504/991] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:09.674272 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373e86e-e2f2-426e-a124-b6d85f166624 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:09.782795 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373e86e-e2f2-426e-a124-b6d85f166624 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:09.818881 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb7fc3-7d0a-45d0-b1a7-f5192886d994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:09.819996 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb7fc3-7d0a-45d0-b1a7-f5192886d994 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:09.820494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb7fc3-7d0a-45d0-b1a7-f5192886d994 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:09.820962 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb7fc3-7d0a-45d0-b1a7-f5192886d994 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:09.821831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb7fc3-7d0a-45d0-b1a7-f5192886d994 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:09.823396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb7fc3-7d0a-45d0-b1a7-f5192886d994 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-76cb7fc3-7d0a-45d0-b1a7-f5192886d994 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:09.836283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:09.837328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:09.838182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:09.838182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:09.838182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:09.838182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:09.838182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:09.838182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:09.838182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:09.838182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:09.838182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:09.838182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:09.838182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:18:09.838182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:09.838182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 15:18:09.838182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:09.838919 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 488/992] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Jul 19 15:18:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:09.846034 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3db1f9-77a0-4fdd-9b9a-8161d366b9ef None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:09.903129 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:09.973396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3db1f9-77a0-4fdd-9b9a-8161d366b9ef None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:10.007993 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e717d4f-6222-42ab-92a2-6fb97cef89b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:10.008660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e717d4f-6222-42ab-92a2-6fb97cef89b2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:10.008944 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e717d4f-6222-42ab-92a2-6fb97cef89b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:10.009247 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e717d4f-6222-42ab-92a2-6fb97cef89b2 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:10.009967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e717d4f-6222-42ab-92a2-6fb97cef89b2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:10.010277 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e717d4f-6222-42ab-92a2-6fb97cef89b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:10.011656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e717d4f-6222-42ab-92a2-6fb97cef89b2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:10.022504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 505/993] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Jul 19 15:18:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:10.023575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb7fc3-7d0a-45d0-b1a7-f5192886d994 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:10.025870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be74b64-7782-4865-b4f2-cb8187048c52 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:10.026161 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be74b64-7782-4865-b4f2-cb8187048c52 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:10.026437 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be74b64-7782-4865-b4f2-cb8187048c52 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:10.027771 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 506/994] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:18:10.032559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c6957-f7af-48e0-98c4-ee42c8fe896f None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:10.032844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c6957-f7af-48e0-98c4-ee42c8fe896f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:10.033126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c6957-f7af-48e0-98c4-ee42c8fe896f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:10.040676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:10.051205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-d41c6957-f7af-48e0-98c4-ee42c8fe896f None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:10.052413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d41c6957-f7af-48e0-98c4-ee42c8fe896f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:10.122715 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 507/995] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:10.131602 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb7fc3-7d0a-45d0-b1a7-f5192886d994 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:10.146653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c6957-f7af-48e0-98c4-ee42c8fe896f None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:10.167904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320638d-3d7b-4459-923d-7172e53610fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:10.168575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320638d-3d7b-4459-923d-7172e53610fb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:10.168865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320638d-3d7b-4459-923d-7172e53610fb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:10.169178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320638d-3d7b-4459-923d-7172e53610fb None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:10.169883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320638d-3d7b-4459-923d-7172e53610fb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:10.170192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320638d-3d7b-4459-923d-7172e53610fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:10.171556 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320638d-3d7b-4459-923d-7172e53610fb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:10.181184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 489/996] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Jul 19 15:18:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:10.182862 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320638d-3d7b-4459-923d-7172e53610fb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:10.187023 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:10.272591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c6957-f7af-48e0-98c4-ee42c8fe896f None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:10.279999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:10.303472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320638d-3d7b-4459-923d-7172e53610fb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:10.307786 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae396b-4241-4242-861d-28be74dae435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:10.308582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae396b-4241-4242-861d-28be74dae435 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:10.308865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae396b-4241-4242-861d-28be74dae435 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:10.309160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae396b-4241-4242-861d-28be74dae435 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:10.309829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae396b-4241-4242-861d-28be74dae435 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:10.311197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae396b-4241-4242-861d-28be74dae435 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-45ae396b-4241-4242-861d-28be74dae435 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:10.323401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:10.324412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:10.325958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:10.325958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:10.325958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:10.325958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:10.325958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:10.325958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:10.325958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:10.325958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:10.325958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:10.325958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:10.325958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:18:10.325958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:10.325958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 15:18:10.325958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:10.325958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 508/997] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:18:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:18:10.326784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe091d-c425-4103-8f6f-0864ea3bc378 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:10.326784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe091d-c425-4103-8f6f-0864ea3bc378 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:10.326784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe091d-c425-4103-8f6f-0864ea3bc378 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:10.327733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 509/998] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:10.331433 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe091d-c425-4103-8f6f-0864ea3bc378 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:10.339360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf228a-7d93-4a84-a47b-92129b58e4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:10.340030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf228a-7d93-4a84-a47b-92129b58e4de None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:10.340322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf228a-7d93-4a84-a47b-92129b58e4de None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:10.340602 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf228a-7d93-4a84-a47b-92129b58e4de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:10.341446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf228a-7d93-4a84-a47b-92129b58e4de None admin] RBAC: Adding query filter params (interface=admin, service_id=3bc9cf4cf5674d4c9b552aaa22e74e07, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:10.341772 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf228a-7d93-4a84-a47b-92129b58e4de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:10.343137 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf228a-7d93-4a84-a47b-92129b58e4de None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:10.352877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 490/999] 10.222.0.42 () {56 vars in 1249 bytes} [Tue Jul 19 15:18:10 2022] GET /identity/v3/endpoints?service_id=3bc9cf4cf5674d4c9b552aaa22e74e07&interface=admin®ion_id=RegionOne => generated 193 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:10.352877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562533d-4c37-46e8-94bb-5779964b901e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:10.352877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562533d-4c37-46e8-94bb-5779964b901e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:10.352877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562533d-4c37-46e8-94bb-5779964b901e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:10.366286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f562533d-4c37-46e8-94bb-5779964b901e None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:10.367475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f562533d-4c37-46e8-94bb-5779964b901e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:10.428908 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:10.433636 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 491/1000] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:10.433854 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe091d-c425-4103-8f6f-0864ea3bc378 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:10.434457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562533d-4c37-46e8-94bb-5779964b901e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:10.480226 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593c377-ead5-4e68-b54d-a4caaa6096b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:10.480919 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593c377-ead5-4e68-b54d-a4caaa6096b0 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:10.481226 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593c377-ead5-4e68-b54d-a4caaa6096b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:10.481499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593c377-ead5-4e68-b54d-a4caaa6096b0 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:10.482212 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593c377-ead5-4e68-b54d-a4caaa6096b0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:10.482521 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593c377-ead5-4e68-b54d-a4caaa6096b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:10.483885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593c377-ead5-4e68-b54d-a4caaa6096b0 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:10.489503 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 510/1001] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Jul 19 15:18:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:10.490300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593c377-ead5-4e68-b54d-a4caaa6096b0 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:10.506563 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:10.537312 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562533d-4c37-46e8-94bb-5779964b901e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:10.571498 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e2f52-ca91-4164-9f93-d861c26e25c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:10.572170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e2f52-ca91-4164-9f93-d861c26e25c1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:10.572436 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e2f52-ca91-4164-9f93-d861c26e25c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:10.572712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e2f52-ca91-4164-9f93-d861c26e25c1 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:10.573422 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e2f52-ca91-4164-9f93-d861c26e25c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:10.574778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e2f52-ca91-4164-9f93-d861c26e25c1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:10.581080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 492/1002] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Jul 19 15:18:10 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:18:10.581953 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e2f52-ca91-4164-9f93-d861c26e25c1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:10.597454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593c377-ead5-4e68-b54d-a4caaa6096b0 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:10.637658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb07d66-dc11-44b7-b2cc-6449dac01ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:10.638480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb07d66-dc11-44b7-b2cc-6449dac01ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:10.638766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb07d66-dc11-44b7-b2cc-6449dac01ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:10.639041 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb07d66-dc11-44b7-b2cc-6449dac01ff9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:10.639713 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb07d66-dc11-44b7-b2cc-6449dac01ff9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:10.641735 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb07d66-dc11-44b7-b2cc-6449dac01ff9 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-5fb07d66-dc11-44b7-b2cc-6449dac01ff9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:10.654230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:10.655505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:10.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:10.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:10.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:10.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:10.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:10.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:10.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:10.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:10.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:10.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:10.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:18:10.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:10.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 15:18:10.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:10.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 511/1003] 10.222.0.42 () {56 vars in 1105 bytes} [Tue Jul 19 15:18:10 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:10.661617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb07d66-dc11-44b7-b2cc-6449dac01ff9 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:10.693680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:10.723058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e2f52-ca91-4164-9f93-d861c26e25c1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:10.765133 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1ae07-451f-4bfe-9520-f907657fa89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:10.766506 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1ae07-451f-4bfe-9520-f907657fa89f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:10.766854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1ae07-451f-4bfe-9520-f907657fa89f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:10.767291 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1ae07-451f-4bfe-9520-f907657fa89f None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:10.768142 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1ae07-451f-4bfe-9520-f907657fa89f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:10.768614 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1ae07-451f-4bfe-9520-f907657fa89f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:10.770652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1ae07-451f-4bfe-9520-f907657fa89f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:10.772376 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:10.777576 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 493/1004] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Jul 19 15:18:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:10.778692 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb07d66-dc11-44b7-b2cc-6449dac01ff9 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:10.785591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1ae07-451f-4bfe-9520-f907657fa89f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:10.818604 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dabb8e-eaa3-492f-be46-af6e8b1093c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:10.819280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dabb8e-eaa3-492f-be46-af6e8b1093c0 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:10.819568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dabb8e-eaa3-492f-be46-af6e8b1093c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:10.819848 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dabb8e-eaa3-492f-be46-af6e8b1093c0 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:10.820569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dabb8e-eaa3-492f-be46-af6e8b1093c0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:10.820882 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dabb8e-eaa3-492f-be46-af6e8b1093c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:10.822300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dabb8e-eaa3-492f-be46-af6e8b1093c0 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:10.827612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 512/1005] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:10.833189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dabb8e-eaa3-492f-be46-af6e8b1093c0 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:10.895700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1ae07-451f-4bfe-9520-f907657fa89f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:10.929736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:10.932503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706ddd6-d72d-4523-b50b-60de223ad743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:10.933194 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706ddd6-d72d-4523-b50b-60de223ad743 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:10.934055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706ddd6-d72d-4523-b50b-60de223ad743 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:10.934336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706ddd6-d72d-4523-b50b-60de223ad743 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:10.935081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706ddd6-d72d-4523-b50b-60de223ad743 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:10.936446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706ddd6-d72d-4523-b50b-60de223ad743 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:10.941606 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dabb8e-eaa3-492f-be46-af6e8b1093c0 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:10.957699 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 494/1006] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Jul 19 15:18:10 2022] POST /identity/v3/endpoints => generated 334 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:18:10.992932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12b81f-9427-48da-8ab5-9b60bad96e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:10.993651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12b81f-9427-48da-8ab5-9b60bad96e36 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:10.993933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12b81f-9427-48da-8ab5-9b60bad96e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:10.994211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12b81f-9427-48da-8ab5-9b60bad96e36 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:10.994941 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12b81f-9427-48da-8ab5-9b60bad96e36 None admin] RBAC: Adding query filter params (type=image) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:10.995250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12b81f-9427-48da-8ab5-9b60bad96e36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:10.996689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12b81f-9427-48da-8ab5-9b60bad96e36 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:11.003451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 513/1007] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:10 2022] GET /identity/v3/services?type=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:12.642814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ce706-c6e2-4365-a993-b2a227de332e None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:12.643328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ce706-c6e2-4365-a993-b2a227de332e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:12.643328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ce706-c6e2-4365-a993-b2a227de332e None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:12.644708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 495/1008] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:12.650100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3ef49-1f2c-43bc-bc0b-2a4b6a9beaa1 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:12.650484 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3ef49-1f2c-43bc-bc0b-2a4b6a9beaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:12.650663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3ef49-1f2c-43bc-bc0b-2a4b6a9beaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:12.668072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-3ac3ef49-1f2c-43bc-bc0b-2a4b6a9beaa1 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:12.669286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3ac3ef49-1f2c-43bc-bc0b-2a4b6a9beaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:12.742892 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 514/1009] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:12 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:12.765597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ce706-c6e2-4365-a993-b2a227de332e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:12.825724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:12.873861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ce706-c6e2-4365-a993-b2a227de332e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:12.909067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d904d-08b8-441f-9b93-3f3566a6af7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:12.909905 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d904d-08b8-441f-9b93-3f3566a6af7c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:12.910192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d904d-08b8-441f-9b93-3f3566a6af7c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:12.910473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d904d-08b8-441f-9b93-3f3566a6af7c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:12.911139 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d904d-08b8-441f-9b93-3f3566a6af7c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:12.912516 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d904d-08b8-441f-9b93-3f3566a6af7c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-667d904d-08b8-441f-9b93-3f3566a6af7c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:12.925314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:12.926307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:12.932444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:12.932444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:12.932444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:12.932444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:12.932444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:12.932444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:12.932444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:12.932444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:12.932444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:12.932444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:12.932444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:18:12.932444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:12.932444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 15:18:12.932444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:12.932444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 496/1010] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Jul 19 15:18:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:18:12.933312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3ef49-1f2c-43bc-bc0b-2a4b6a9beaa1 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:13.014731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:13.036441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3ef49-1f2c-43bc-bc0b-2a4b6a9beaa1 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:13.055309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff6186-4b89-4481-9eae-8d7efe1b04ed None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:13.055618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff6186-4b89-4481-9eae-8d7efe1b04ed None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:13.055859 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff6186-4b89-4481-9eae-8d7efe1b04ed None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:13.057385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 497/1011] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:13.062134 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca1f53-33cf-4950-94d9-b544c0c6f41a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:13.062481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca1f53-33cf-4950-94d9-b544c0c6f41a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:13.062835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca1f53-33cf-4950-94d9-b544c0c6f41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:13.073926 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b506c-d273-47bb-ab8c-ad3f5db9543b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:13.074620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b506c-d273-47bb-ab8c-ad3f5db9543b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:13.075371 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b506c-d273-47bb-ab8c-ad3f5db9543b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:13.075651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b506c-d273-47bb-ab8c-ad3f5db9543b None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:13.076639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b506c-d273-47bb-ab8c-ad3f5db9543b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:13.076951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b506c-d273-47bb-ab8c-ad3f5db9543b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:13.078992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b506c-d273-47bb-ab8c-ad3f5db9543b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:13.080760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-36ca1f53-33cf-4950-94d9-b544c0c6f41a None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:13.081998 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-36ca1f53-33cf-4950-94d9-b544c0c6f41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:13.085449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 515/1012] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Jul 19 15:18:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:13.090053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b506c-d273-47bb-ab8c-ad3f5db9543b None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:13.165392 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 498/1013] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:13 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:18:13.166728 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f53a9-731f-4451-929e-0e3746ab0e79 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:13.167020 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f53a9-731f-4451-929e-0e3746ab0e79 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:13.167303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f53a9-731f-4451-929e-0e3746ab0e79 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:13.168738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 499/1014] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:13.174914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2a20c-3aa0-4cd5-b781-812a7e9cce9b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:13.175200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2a20c-3aa0-4cd5-b781-812a7e9cce9b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:13.175480 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2a20c-3aa0-4cd5-b781-812a7e9cce9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:13.192537 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-34c2a20c-3aa0-4cd5-b781-812a7e9cce9b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:13.193750 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-34c2a20c-3aa0-4cd5-b781-812a7e9cce9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:13.196646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b506c-d273-47bb-ab8c-ad3f5db9543b None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:13.231373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178d49f-a0fc-4371-a700-da32717fc8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:13.232035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178d49f-a0fc-4371-a700-da32717fc8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:13.232314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178d49f-a0fc-4371-a700-da32717fc8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:13.232586 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178d49f-a0fc-4371-a700-da32717fc8e4 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:13.233328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178d49f-a0fc-4371-a700-da32717fc8e4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:13.233635 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178d49f-a0fc-4371-a700-da32717fc8e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:13.235032 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178d49f-a0fc-4371-a700-da32717fc8e4 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:13.242591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 516/1015] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Jul 19 15:18:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:13.243365 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178d49f-a0fc-4371-a700-da32717fc8e4 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:13.243655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:13.255007 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:13.268383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 500/1016] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:13 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:18:13.270173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2a20c-3aa0-4cd5-b781-812a7e9cce9b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:13.350372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178d49f-a0fc-4371-a700-da32717fc8e4 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:13.372015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2a20c-3aa0-4cd5-b781-812a7e9cce9b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:13.384013 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb787ffd-4264-4fdc-a022-aead16f97013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:13.384672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb787ffd-4264-4fdc-a022-aead16f97013 None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:13.384953 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb787ffd-4264-4fdc-a022-aead16f97013 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:13.385268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb787ffd-4264-4fdc-a022-aead16f97013 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:13.385988 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb787ffd-4264-4fdc-a022-aead16f97013 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:13.387367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb787ffd-4264-4fdc-a022-aead16f97013 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:13.402176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 517/1017] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Jul 19 15:18:13 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:18:13.402961 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb787ffd-4264-4fdc-a022-aead16f97013 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:13.417899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b97f5-62e3-4a2a-a9fe-85d42a7f04f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:13.418507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b97f5-62e3-4a2a-a9fe-85d42a7f04f6 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:13.418785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b97f5-62e3-4a2a-a9fe-85d42a7f04f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:13.419061 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b97f5-62e3-4a2a-a9fe-85d42a7f04f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:13.419836 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b97f5-62e3-4a2a-a9fe-85d42a7f04f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:13.421338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b97f5-62e3-4a2a-a9fe-85d42a7f04f6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:13.440110 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 501/1018] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Jul 19 15:18:13 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:18:13.503858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:13.551753 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb787ffd-4264-4fdc-a022-aead16f97013 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:13.602370 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44a27b-32d0-4aac-99d6-e6e178ac06f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:13.603187 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44a27b-32d0-4aac-99d6-e6e178ac06f6 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:13.603475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44a27b-32d0-4aac-99d6-e6e178ac06f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:13.603744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44a27b-32d0-4aac-99d6-e6e178ac06f6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:13.616194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44a27b-32d0-4aac-99d6-e6e178ac06f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-9e44a27b-32d0-4aac-99d6-e6e178ac06f6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:18:13.624435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:13.625736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:13.626778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:13.626778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:13.626778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:13.626778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:13.626778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:18:13.626778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:18:13.626778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:18:13.626778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:18:13.626778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:18:13.626778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:18:13.626778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 15:18:13.626778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:13.626778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 518/1019] 10.222.0.42 () {56 vars in 1120 bytes} [Tue Jul 19 15:18:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:13.631986 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b97f5-62e3-4a2a-a9fe-85d42a7f04f6 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:13.669629 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:13.741404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b97f5-62e3-4a2a-a9fe-85d42a7f04f6 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:13.776030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a926d58-daf5-458c-83c0-6ecf02da44e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:13.776971 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a926d58-daf5-458c-83c0-6ecf02da44e7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:13.777431 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a926d58-daf5-458c-83c0-6ecf02da44e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:13.777851 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a926d58-daf5-458c-83c0-6ecf02da44e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:13.778748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a926d58-daf5-458c-83c0-6ecf02da44e7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:13.779221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a926d58-daf5-458c-83c0-6ecf02da44e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:13.780716 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a926d58-daf5-458c-83c0-6ecf02da44e7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:13.790090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 502/1020] 10.222.0.42 () {56 vars in 1115 bytes} [Tue Jul 19 15:18:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:13.795472 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44a27b-32d0-4aac-99d6-e6e178ac06f6 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:13.899856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44a27b-32d0-4aac-99d6-e6e178ac06f6 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:13.924452 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:13.939072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a58238-6bbe-422b-988b-d0e24557d983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:13.939887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a58238-6bbe-422b-988b-d0e24557d983 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:13.940287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a58238-6bbe-422b-988b-d0e24557d983 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:13.940607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a58238-6bbe-422b-988b-d0e24557d983 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:13.952018 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a58238-6bbe-422b-988b-d0e24557d983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:13.953895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a58238-6bbe-422b-988b-d0e24557d983 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:13.971602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-c2a58238-6bbe-422b-988b-d0e24557d983 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:13.971602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:13.971602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:13.971602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:13.971602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:13.971602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:13.971602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:13.971602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:13.971602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:13.971602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:13.971602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:13.971602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:13.971602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:18:13.971602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:18:13.971602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:13.972533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:13.973474 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:18:13.973474 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:18:13.973474 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:13.973474 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:13.977473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 519/1021] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:13.978455 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a926d58-daf5-458c-83c0-6ecf02da44e7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:14.075071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:14.082538 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a926d58-daf5-458c-83c0-6ecf02da44e7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:14.117088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d5d72-6f72-4612-b180-00a39de2d6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:14.117783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d5d72-6f72-4612-b180-00a39de2d6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:14.118072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d5d72-6f72-4612-b180-00a39de2d6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:14.118353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d5d72-6f72-4612-b180-00a39de2d6a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:14.119086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d5d72-6f72-4612-b180-00a39de2d6a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:14.119399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d5d72-6f72-4612-b180-00a39de2d6a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:14.120769 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d5d72-6f72-4612-b180-00a39de2d6a8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:14.133978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 503/1022] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:14.139368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a58238-6bbe-422b-988b-d0e24557d983 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:14.254242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a58238-6bbe-422b-988b-d0e24557d983 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:14.291513 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02300ddf-c866-4d36-a286-15fa19092ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:14.292301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02300ddf-c866-4d36-a286-15fa19092ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:14.292581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02300ddf-c866-4d36-a286-15fa19092ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:14.292911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02300ddf-c866-4d36-a286-15fa19092ce5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:14.301915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02300ddf-c866-4d36-a286-15fa19092ce5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:14.303340 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02300ddf-c866-4d36-a286-15fa19092ce5 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-02300ddf-c866-4d36-a286-15fa19092ce5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:18:14.317954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:14.318885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:14.319867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:14.319867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:14.319867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:18:14.319867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:18:14.319867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:18:14.319867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:18:14.319867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:18:14.319867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:18:14.319867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:18:14.319867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:14.321311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 520/1023] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Jul 19 15:18:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:18:14.324838 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d5d72-6f72-4612-b180-00a39de2d6a8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:14.426503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d5d72-6f72-4612-b180-00a39de2d6a8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:14.460065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8b563-102b-43ed-9bd8-ec49c7160728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:14.460734 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8b563-102b-43ed-9bd8-ec49c7160728 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:14.461015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8b563-102b-43ed-9bd8-ec49c7160728 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:14.461319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8b563-102b-43ed-9bd8-ec49c7160728 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:14.462152 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8b563-102b-43ed-9bd8-ec49c7160728 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:14.462459 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8b563-102b-43ed-9bd8-ec49c7160728 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:14.463866 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8b563-102b-43ed-9bd8-ec49c7160728 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:14.482096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 504/1024] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Jul 19 15:18:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:14.488024 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02300ddf-c866-4d36-a286-15fa19092ce5 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:14.524352 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:14.598683 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02300ddf-c866-4d36-a286-15fa19092ce5 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:14.645316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1e441-0600-42ca-90bc-b9d78c27ad33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:14.645970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1e441-0600-42ca-90bc-b9d78c27ad33 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:14.646253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1e441-0600-42ca-90bc-b9d78c27ad33 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:14.646529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1e441-0600-42ca-90bc-b9d78c27ad33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:14.715909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1e441-0600-42ca-90bc-b9d78c27ad33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:14.717390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1e441-0600-42ca-90bc-b9d78c27ad33 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:14.753432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:14.819142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 521/1025] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:18:14 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:18:14.824541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8b563-102b-43ed-9bd8-ec49c7160728 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:14.840956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:14.927954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8b563-102b-43ed-9bd8-ec49c7160728 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:14.964444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4177e6-4e19-4779-8643-259074421aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:14.965277 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4177e6-4e19-4779-8643-259074421aec None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:14.965579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4177e6-4e19-4779-8643-259074421aec None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:14.965864 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4177e6-4e19-4779-8643-259074421aec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:14.976171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4177e6-4e19-4779-8643-259074421aec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:14.977635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4177e6-4e19-4779-8643-259074421aec None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:14.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-de4177e6-4e19-4779-8643-259074421aec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:14.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:14.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:14.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:14.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:14.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:14.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:14.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:14.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:14.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:14.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:14.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:14.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:18:14.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:18:14.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:14.995972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:14.996912 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:18:14.996912 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:18:14.996912 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:14.996912 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:14.997891 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 505/1026] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:15.002852 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1e441-0600-42ca-90bc-b9d78c27ad33 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:15.106986 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1e441-0600-42ca-90bc-b9d78c27ad33 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:15.142486 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb9da7-6580-4680-9c2b-9888a9bde43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:15.143496 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb9da7-6580-4680-9c2b-9888a9bde43f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:15.144029 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb9da7-6580-4680-9c2b-9888a9bde43f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:15.144525 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb9da7-6580-4680-9c2b-9888a9bde43f None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:15.145487 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb9da7-6580-4680-9c2b-9888a9bde43f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:15.145992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb9da7-6580-4680-9c2b-9888a9bde43f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:15.147548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb9da7-6580-4680-9c2b-9888a9bde43f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:15.150462 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:15.166030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 522/1027] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:15.167040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4177e6-4e19-4779-8643-259074421aec None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:15.235301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:15.277930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4177e6-4e19-4779-8643-259074421aec None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:15.315252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d8804-58ce-438a-97b0-79fb3769ba63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:15.316051 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d8804-58ce-438a-97b0-79fb3769ba63 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:15.316336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d8804-58ce-438a-97b0-79fb3769ba63 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:15.316614 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d8804-58ce-438a-97b0-79fb3769ba63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:15.323487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d8804-58ce-438a-97b0-79fb3769ba63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:15.324934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d8804-58ce-438a-97b0-79fb3769ba63 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:15.339193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-570d8804-58ce-438a-97b0-79fb3769ba63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:18:15.339193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:15.339193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:15.339193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:15.339193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:15.339193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:15.339193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:15.339193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:15.339193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:15.339193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:15.339193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:15.339193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:15.339193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:18:15.339193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:18:15.339193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:15.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:15.340988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:18:15.340988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:18:15.340988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:18:15.340988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:18:15.340988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:18:15.340988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:18:15.340988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:18:15.340988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:15.342170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 506/1028] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:18:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:15.346804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb9da7-6580-4680-9c2b-9888a9bde43f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:15.450178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb9da7-6580-4680-9c2b-9888a9bde43f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:15.483534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0d594-7f75-4a0d-89f3-8c99187b90da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:15.484824 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0d594-7f75-4a0d-89f3-8c99187b90da None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:15.485187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abe964-a9a9-4bed-939c-d898337a42df None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:15.485297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0d594-7f75-4a0d-89f3-8c99187b90da None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:15.485548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abe964-a9a9-4bed-939c-d898337a42df None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:15.485666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0d594-7f75-4a0d-89f3-8c99187b90da None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:15.485829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abe964-a9a9-4bed-939c-d898337a42df None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:15.486313 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0d594-7f75-4a0d-89f3-8c99187b90da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:15.486622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0d594-7f75-4a0d-89f3-8c99187b90da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:15.487284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 507/1029] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:15.488167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0d594-7f75-4a0d-89f3-8c99187b90da None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:15.492047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90db1f3-301b-4789-a266-655483a66a4c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:15.492340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90db1f3-301b-4789-a266-655483a66a4c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:15.492621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90db1f3-301b-4789-a266-655483a66a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:15.507366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 523/1030] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:15.508386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0d594-7f75-4a0d-89f3-8c99187b90da None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:15.516325 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d90db1f3-301b-4789-a266-655483a66a4c None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:15.517547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d90db1f3-301b-4789-a266-655483a66a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:15.568167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:15.589013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 508/1031] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:18:15.590385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84b1bd-8215-479f-a8e2-0eeaef7fb871 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:15.590674 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84b1bd-8215-479f-a8e2-0eeaef7fb871 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:15.590955 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84b1bd-8215-479f-a8e2-0eeaef7fb871 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:15.593283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 509/1032] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:15.598024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e820d5-ea3b-4137-935e-17c9119022d3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:15.598349 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e820d5-ea3b-4137-935e-17c9119022d3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:15.598732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e820d5-ea3b-4137-935e-17c9119022d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:15.614821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e2e820d5-ea3b-4137-935e-17c9119022d3 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:15.615551 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0d594-7f75-4a0d-89f3-8c99187b90da None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:15.615996 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e2e820d5-ea3b-4137-935e-17c9119022d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:15.651481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d0d31-ff5b-4503-8187-a513a0c93296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:15.652132 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d0d31-ff5b-4503-8187-a513a0c93296 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:15.652417 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d0d31-ff5b-4503-8187-a513a0c93296 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:15.652691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d0d31-ff5b-4503-8187-a513a0c93296 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:15.653559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d0d31-ff5b-4503-8187-a513a0c93296 None admin] RBAC: Adding query filter params (role.id=d7b232d8f46040ba8eab9538a784e61f, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=bff6a2d4bc734904b2615608a86b097d) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:15.653869 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d0d31-ff5b-4503-8187-a513a0c93296 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:15.655303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d0d31-ff5b-4503-8187-a513a0c93296 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:15.662120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 524/1033] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:18:15 2022] GET /identity/v3/role_assignments?user.id=bff6a2d4bc734904b2615608a86b097d&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=d7b232d8f46040ba8eab9538a784e61f => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:15.663027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d0d31-ff5b-4503-8187-a513a0c93296 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:15.668718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:15.692297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 510/1034] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:18:15.716764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e820d5-ea3b-4137-935e-17c9119022d3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:15.780386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d0d31-ff5b-4503-8187-a513a0c93296 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:15.805458 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:15.832504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808e653-3bd9-4da0-91aa-d1a4a4ab5033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:15.832504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808e653-3bd9-4da0-91aa-d1a4a4ab5033 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:15.832504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808e653-3bd9-4da0-91aa-d1a4a4ab5033 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:15.832504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808e653-3bd9-4da0-91aa-d1a4a4ab5033 None admin] PATH_INFO: `/v3/services/image` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:15.833444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1808e653-3bd9-4da0-91aa-d1a4a4ab5033 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:15.833444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1808e653-3bd9-4da0-91aa-d1a4a4ab5033 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:15.833611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e820d5-ea3b-4137-935e-17c9119022d3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-1808e653-3bd9-4da0-91aa-d1a4a4ab5033 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:15.844949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:15.846249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:15.847287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:15.847287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:15.847287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:15.847287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:15.847287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:15.847287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:15.847287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:15.847287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:15.847287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:15.847287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:15.847287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:18:15.847287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:15.847287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 15:18:15.847287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:15.853170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 525/1035] 10.222.0.42 () {56 vars in 1105 bytes} [Tue Jul 19 15:18:15 2022] GET /identity/v3/services/image => generated 86 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:15.853170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808e653-3bd9-4da0-91aa-d1a4a4ab5033 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:15.869326 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e486b4-9288-4526-b735-2fe5b673a1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:15.870129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e486b4-9288-4526-b735-2fe5b673a1ba None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:15.870414 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e486b4-9288-4526-b735-2fe5b673a1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:15.870692 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e486b4-9288-4526-b735-2fe5b673a1ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:15.871873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e486b4-9288-4526-b735-2fe5b673a1ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:15.873325 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e486b4-9288-4526-b735-2fe5b673a1ba None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:15.880359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 511/1036] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:15.881843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e486b4-9288-4526-b735-2fe5b673a1ba None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:15.945845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:15.961310 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808e653-3bd9-4da0-91aa-d1a4a4ab5033 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:15.999194 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e486b4-9288-4526-b735-2fe5b673a1ba None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:16.001576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee790674-fb03-414d-a5da-6af3ea05c37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:16.002249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee790674-fb03-414d-a5da-6af3ea05c37e None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:16.002534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee790674-fb03-414d-a5da-6af3ea05c37e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:16.002818 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee790674-fb03-414d-a5da-6af3ea05c37e None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:16.003533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee790674-fb03-414d-a5da-6af3ea05c37e None admin] RBAC: Adding query filter params (name=image) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:16.003843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee790674-fb03-414d-a5da-6af3ea05c37e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:16.005245 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee790674-fb03-414d-a5da-6af3ea05c37e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:16.014947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 526/1037] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:16.016237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee790674-fb03-414d-a5da-6af3ea05c37e None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:16.045028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c02988-3409-4e20-bcfb-27d4aaf83f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:16.045752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c02988-3409-4e20-bcfb-27d4aaf83f35 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:16.046039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c02988-3409-4e20-bcfb-27d4aaf83f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:16.046313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c02988-3409-4e20-bcfb-27d4aaf83f35 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:16.048024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c02988-3409-4e20-bcfb-27d4aaf83f35 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:16.049491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c02988-3409-4e20-bcfb-27d4aaf83f35 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:16.068573 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:16.080432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 512/1038] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Jul 19 15:18:15 2022] POST /identity/v3/projects => generated 311 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:18:16.141579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee790674-fb03-414d-a5da-6af3ea05c37e None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:16.179626 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af19335-e90b-412a-a6d2-2d7e2043dfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:16.180306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af19335-e90b-412a-a6d2-2d7e2043dfec None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:16.180610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af19335-e90b-412a-a6d2-2d7e2043dfec None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:16.180869 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af19335-e90b-412a-a6d2-2d7e2043dfec None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:16.181632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af19335-e90b-412a-a6d2-2d7e2043dfec None admin] RBAC: Adding query filter params (type=image) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:16.181948 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af19335-e90b-412a-a6d2-2d7e2043dfec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:16.183404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af19335-e90b-412a-a6d2-2d7e2043dfec None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:16.193914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 527/1039] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:16.195250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c02988-3409-4e20-bcfb-27d4aaf83f35 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:16.308827 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c02988-3409-4e20-bcfb-27d4aaf83f35 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:16.316819 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:16.346187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b028e-6424-4303-b876-e969f93d2eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:16.346853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b028e-6424-4303-b876-e969f93d2eea None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:16.347139 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b028e-6424-4303-b876-e969f93d2eea None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:16.347420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b028e-6424-4303-b876-e969f93d2eea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:16.348226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b028e-6424-4303-b876-e969f93d2eea None admin] RBAC: Adding query filter params (interface=public, service_id=c1021597ec7d42efbb7570dbde2b6d6e, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:16.348542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b028e-6424-4303-b876-e969f93d2eea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:16.349957 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b028e-6424-4303-b876-e969f93d2eea None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:16.355208 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 513/1040] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Jul 19 15:18:16 2022] GET /identity/v3/endpoints?service_id=c1021597ec7d42efbb7570dbde2b6d6e&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:16.361324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af19335-e90b-412a-a6d2-2d7e2043dfec None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:16.462946 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af19335-e90b-412a-a6d2-2d7e2043dfec None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:16.476459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:16.498909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccced090-9b6a-472a-a997-60a0d5741aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:16.499576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccced090-9b6a-472a-a997-60a0d5741aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:16.499894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccced090-9b6a-472a-a997-60a0d5741aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:16.500174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccced090-9b6a-472a-a997-60a0d5741aa5 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:16.500872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccced090-9b6a-472a-a997-60a0d5741aa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:16.502277 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccced090-9b6a-472a-a997-60a0d5741aa5 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:16.517097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 528/1041] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Jul 19 15:18:16 2022] GET /identity/v3/services => generated 1957 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:18:17.714079 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c142f15-c79a-42be-81db-663219579aaf None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:17.714573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c142f15-c79a-42be-81db-663219579aaf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:17.714573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c142f15-c79a-42be-81db-663219579aaf None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:17.715974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 514/1042] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:17.721347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04759068-97dc-44ae-9095-f2740dcf77e2 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:17.721758 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04759068-97dc-44ae-9095-f2740dcf77e2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:17.721929 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04759068-97dc-44ae-9095-f2740dcf77e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:17.739366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-04759068-97dc-44ae-9095-f2740dcf77e2 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:17.740554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-04759068-97dc-44ae-9095-f2740dcf77e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:17.809473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 529/1043] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:17.832572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c142f15-c79a-42be-81db-663219579aaf None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:17.936999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c142f15-c79a-42be-81db-663219579aaf None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:17.972997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238bfde-4e39-42e0-92f9-b50638bcbf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:17.973845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238bfde-4e39-42e0-92f9-b50638bcbf52 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:17.974127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238bfde-4e39-42e0-92f9-b50638bcbf52 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:17.974399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238bfde-4e39-42e0-92f9-b50638bcbf52 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:17.985173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238bfde-4e39-42e0-92f9-b50638bcbf52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3238bfde-4e39-42e0-92f9-b50638bcbf52 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:18:17.993246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:17.994184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:17.995028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:17.995028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:17.995028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:17.995028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:17.995028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:18:17.995028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:18:17.995028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:18:17.995028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:18:17.995028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:18:17.995028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:18:17.995028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 15:18:17.995028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:17.997491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 515/1044] 10.222.0.42 () {56 vars in 1120 bytes} [Tue Jul 19 15:18:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:18.001055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04759068-97dc-44ae-9095-f2740dcf77e2 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:18.104669 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04759068-97dc-44ae-9095-f2740dcf77e2 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:18.113690 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:18.141651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4029a27-c158-49aa-80f2-e0f25fde9d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:18.142328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4029a27-c158-49aa-80f2-e0f25fde9d82 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:18.142611 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4029a27-c158-49aa-80f2-e0f25fde9d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:18.142885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4029a27-c158-49aa-80f2-e0f25fde9d82 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:18.143638 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4029a27-c158-49aa-80f2-e0f25fde9d82 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:18.143945 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4029a27-c158-49aa-80f2-e0f25fde9d82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:18.145330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4029a27-c158-49aa-80f2-e0f25fde9d82 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:18.158905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 530/1045] 10.222.0.42 () {56 vars in 1115 bytes} [Tue Jul 19 15:18:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:18.160360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238bfde-4e39-42e0-92f9-b50638bcbf52 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:18.172136 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270bcb9-ca61-4964-8421-9d95f0940328 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:18.172551 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270bcb9-ca61-4964-8421-9d95f0940328 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:18.172674 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270bcb9-ca61-4964-8421-9d95f0940328 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:18.174182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 531/1046] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:18:18.179012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41cac8-b1ec-4d0a-b89a-cc0a6c3a03f8 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:18.179295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41cac8-b1ec-4d0a-b89a-cc0a6c3a03f8 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:18.179578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41cac8-b1ec-4d0a-b89a-cc0a6c3a03f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:18.195382 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:18.197695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-fb41cac8-b1ec-4d0a-b89a-cc0a6c3a03f8 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:18.199043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-fb41cac8-b1ec-4d0a-b89a-cc0a6c3a03f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:18.276697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238bfde-4e39-42e0-92f9-b50638bcbf52 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:18.276977 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 532/1047] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:18:18.302491 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41cac8-b1ec-4d0a-b89a-cc0a6c3a03f8 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:18.314200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342ee40-3526-4508-a0e6-52e121e079b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:18.315002 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342ee40-3526-4508-a0e6-52e121e079b3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:18.315286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342ee40-3526-4508-a0e6-52e121e079b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:18.315564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342ee40-3526-4508-a0e6-52e121e079b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:18.327452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342ee40-3526-4508-a0e6-52e121e079b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:18.330001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342ee40-3526-4508-a0e6-52e121e079b3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:18.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-4342ee40-3526-4508-a0e6-52e121e079b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:18.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:18.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:18.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:18.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:18.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:18.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:18.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:18.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:18.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:18.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:18.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:18.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:18:18.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:18:18.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:18.353063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:18.354002 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:18:18.354002 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:18:18.354002 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:18.354002 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:18.354233 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 516/1048] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:18:18.360701 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342ee40-3526-4508-a0e6-52e121e079b3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:18.412476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:18.417724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41cac8-b1ec-4d0a-b89a-cc0a6c3a03f8 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:18.455047 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9035af7-4693-46fd-9910-e8a08a8bf91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:18.455848 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9035af7-4693-46fd-9910-e8a08a8bf91a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:18.456128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9035af7-4693-46fd-9910-e8a08a8bf91a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:18.456624 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9035af7-4693-46fd-9910-e8a08a8bf91a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:18.457316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:18.457621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9035af7-4693-46fd-9910-e8a08a8bf91a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:18.459561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9035af7-4693-46fd-9910-e8a08a8bf91a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:18.465220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 533/1049] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:18.467162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9035af7-4693-46fd-9910-e8a08a8bf91a None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:18.477440 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342ee40-3526-4508-a0e6-52e121e079b3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:18.517891 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fd972-645f-42ed-bfaa-54da1527f907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:18.518482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fd972-645f-42ed-bfaa-54da1527f907 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:18.518767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fd972-645f-42ed-bfaa-54da1527f907 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:18.519048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fd972-645f-42ed-bfaa-54da1527f907 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:18.519776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fd972-645f-42ed-bfaa-54da1527f907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:18.520083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fd972-645f-42ed-bfaa-54da1527f907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:18.521525 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fd972-645f-42ed-bfaa-54da1527f907 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:18.535340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 517/1050] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:18.541129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fd972-645f-42ed-bfaa-54da1527f907 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:18.577161 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9035af7-4693-46fd-9910-e8a08a8bf91a None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:18.612209 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a825cff-1df1-4dbb-8019-d00b55f69933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:18.612884 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a825cff-1df1-4dbb-8019-d00b55f69933 None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:18.613184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a825cff-1df1-4dbb-8019-d00b55f69933 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:18.613464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a825cff-1df1-4dbb-8019-d00b55f69933 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:18.614179 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a825cff-1df1-4dbb-8019-d00b55f69933 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:18.615621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a825cff-1df1-4dbb-8019-d00b55f69933 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:18.635603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 534/1051] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:18:18 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:18:18.637011 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a0ec5-e9f6-4dc7-99c6-192203807ff0 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:18.637331 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a0ec5-e9f6-4dc7-99c6-192203807ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:18.637612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a0ec5-e9f6-4dc7-99c6-192203807ff0 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:18.639005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 535/1052] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:18.651003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40a0ee-f455-4f75-ac91-091e907a8134 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:18.651357 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40a0ee-f455-4f75-ac91-091e907a8134 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:18.651553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40a0ee-f455-4f75-ac91-091e907a8134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:18.651990 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fd972-645f-42ed-bfaa-54da1527f907 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:18.670658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-9b40a0ee-f455-4f75-ac91-091e907a8134 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:18.671989 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9b40a0ee-f455-4f75-ac91-091e907a8134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:18.692389 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:18.699479 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61e528-4d24-47b7-94e2-9e0161b2f04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:18.700626 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61e528-4d24-47b7-94e2-9e0161b2f04e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:18.701151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61e528-4d24-47b7-94e2-9e0161b2f04e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:18.701538 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61e528-4d24-47b7-94e2-9e0161b2f04e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:18.710817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61e528-4d24-47b7-94e2-9e0161b2f04e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:18.712312 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61e528-4d24-47b7-94e2-9e0161b2f04e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-df61e528-4d24-47b7-94e2-9e0161b2f04e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:18:18.728108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:18.729644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:18.731615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:18.731615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:18.731615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:18:18.731615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:18:18.731615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:18:18.731615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:18:18.731615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:18:18.731615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:18:18.731615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:18:18.731615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:18.731615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 518/1053] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Jul 19 15:18:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:18:18.735531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61e528-4d24-47b7-94e2-9e0161b2f04e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:18.738671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:18.762480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 536/1054] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:18:18.789276 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40a0ee-f455-4f75-ac91-091e907a8134 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:18.869807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61e528-4d24-47b7-94e2-9e0161b2f04e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:18.904929 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40a0ee-f455-4f75-ac91-091e907a8134 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:18.908682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6320108-f81a-43d2-9c4f-c325936e9c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:18.909391 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6320108-f81a-43d2-9c4f-c325936e9c1b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:18.909680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6320108-f81a-43d2-9c4f-c325936e9c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:18.909959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6320108-f81a-43d2-9c4f-c325936e9c1b None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:18.910810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6320108-f81a-43d2-9c4f-c325936e9c1b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:18.911123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6320108-f81a-43d2-9c4f-c325936e9c1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:18.912585 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6320108-f81a-43d2-9c4f-c325936e9c1b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:18.930929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 519/1055] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Jul 19 15:18:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:18.936903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6320108-f81a-43d2-9c4f-c325936e9c1b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:18.946104 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8f6ed-f45f-4cf0-acb4-a8df2bf373ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:18.947318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8f6ed-f45f-4cf0-acb4-a8df2bf373ac None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:18.947605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8f6ed-f45f-4cf0-acb4-a8df2bf373ac None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:18.947880 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8f6ed-f45f-4cf0-acb4-a8df2bf373ac None admin] PATH_INFO: `/v3/services/image` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:18.948869 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8f6ed-f45f-4cf0-acb4-a8df2bf373ac None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:18.950639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8f6ed-f45f-4cf0-acb4-a8df2bf373ac None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-15e8f6ed-f45f-4cf0-acb4-a8df2bf373ac None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:18.963130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:18.964193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:18.965165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:18.965165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:18.965165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:18.965165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:18.965165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:18.965165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:18.965165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:18.965165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:18.965165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:18.965165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:18.965165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:18:18.965165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:18:18.965165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 15:18:18.965165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:18.965165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 537/1056] 10.222.0.42 () {56 vars in 1105 bytes} [Tue Jul 19 15:18:18 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:18:18.970956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8f6ed-f45f-4cf0-acb4-a8df2bf373ac None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:18.990819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:19.004959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:19.050122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6320108-f81a-43d2-9c4f-c325936e9c1b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:19.082830 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8f6ed-f45f-4cf0-acb4-a8df2bf373ac None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:19.088275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c47bc8-2819-43e3-8187-7c8525328bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:19.089063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c47bc8-2819-43e3-8187-7c8525328bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:19.089380 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c47bc8-2819-43e3-8187-7c8525328bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:19.089713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c47bc8-2819-43e3-8187-7c8525328bd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:19.101504 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c47bc8-2819-43e3-8187-7c8525328bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:19.103083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c47bc8-2819-43e3-8187-7c8525328bd5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:19.117569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868b098-6fc4-443c-8804-47c46bd67f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:19.118239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868b098-6fc4-443c-8804-47c46bd67f9d None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:19.118521 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868b098-6fc4-443c-8804-47c46bd67f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:19.118796 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868b098-6fc4-443c-8804-47c46bd67f9d None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:19.119517 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868b098-6fc4-443c-8804-47c46bd67f9d None admin] RBAC: Adding query filter params (name=image) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:19.119823 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868b098-6fc4-443c-8804-47c46bd67f9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:19.121230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868b098-6fc4-443c-8804-47c46bd67f9d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:19.122371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-80c47bc8-2819-43e3-8187-7c8525328bd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:19.122371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:19.122371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:19.122371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:19.122371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:19.122371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:19.122371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:19.122371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:19.122371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:19.122371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:19.122371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:19.122371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:19.122371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:18:19.122371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:18:19.122371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:19.123207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:19.124096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:18:19.124096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:18:19.124096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:19.124096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:19.124096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 520/1057] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:18:19.129520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c47bc8-2819-43e3-8187-7c8525328bd5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:19.131985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 538/1058] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:19.137421 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868b098-6fc4-443c-8804-47c46bd67f9d None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:19.221149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:19.243809 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868b098-6fc4-443c-8804-47c46bd67f9d None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:19.250775 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c47bc8-2819-43e3-8187-7c8525328bd5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:19.264847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:19.282603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d340d89-8520-4db1-8fec-87257fe73bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:19.283274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d340d89-8520-4db1-8fec-87257fe73bcc None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:19.283563 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d340d89-8520-4db1-8fec-87257fe73bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:19.283848 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d340d89-8520-4db1-8fec-87257fe73bcc None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:19.284559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d340d89-8520-4db1-8fec-87257fe73bcc None admin] RBAC: Adding query filter params (type=image) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:19.284870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d340d89-8520-4db1-8fec-87257fe73bcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:19.286288 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d340d89-8520-4db1-8fec-87257fe73bcc None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:19.289001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd6530-98ce-4868-8eb7-677d57c50f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:19.289690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd6530-98ce-4868-8eb7-677d57c50f0d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:19.289972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd6530-98ce-4868-8eb7-677d57c50f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:19.290246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd6530-98ce-4868-8eb7-677d57c50f0d None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:19.290962 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd6530-98ce-4868-8eb7-677d57c50f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:19.291472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd6530-98ce-4868-8eb7-677d57c50f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:19.291577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 539/1059] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:19.292762 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd6530-98ce-4868-8eb7-677d57c50f0d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:19.299958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d340d89-8520-4db1-8fec-87257fe73bcc None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:19.310695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 521/1060] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:19.311778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd6530-98ce-4868-8eb7-677d57c50f0d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:19.420756 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d340d89-8520-4db1-8fec-87257fe73bcc None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:19.423464 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd6530-98ce-4868-8eb7-677d57c50f0d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:19.458165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fdc789-e4bd-4c00-aab5-8a045fe79c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:19.458460 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b1fb7-bc14-401f-8fa2-7882aa6f7b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:19.458982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdc789-e4bd-4c00-aab5-8a045fe79c94 None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:19.459272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdc789-e4bd-4c00-aab5-8a045fe79c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:19.459380 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b1fb7-bc14-401f-8fa2-7882aa6f7b47 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:19.459556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdc789-e4bd-4c00-aab5-8a045fe79c94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:19.459652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b1fb7-bc14-401f-8fa2-7882aa6f7b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:19.459942 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b1fb7-bc14-401f-8fa2-7882aa6f7b47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:19.460301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fdc789-e4bd-4c00-aab5-8a045fe79c94 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:19.461911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fdc789-e4bd-4c00-aab5-8a045fe79c94 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:19.467132 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b1fb7-bc14-401f-8fa2-7882aa6f7b47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:19.469366 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b1fb7-bc14-401f-8fa2-7882aa6f7b47 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:19.478012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:19.484711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f50b1fb7-bc14-401f-8fa2-7882aa6f7b47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:18:19.484711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:19.484711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:19.484711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:19.484711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:19.484711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:19.484711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:19.484711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:19.484711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:19.484711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:19.484711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:19.484711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:19.484711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:18:19.484711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:18:19.484711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:19.485649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:19.491930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:18:19.491930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:18:19.491930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:18:19.491930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:18:19.491930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:18:19.491930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:18:19.491930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:18:19.491930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:19.491930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 522/1061] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:18:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:19.492437 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b1fb7-bc14-401f-8fa2-7882aa6f7b47 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:19.493350 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 540/1062] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Jul 19 15:18:19 2022] POST /identity/v3/endpoints => generated 327 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:18:19.526595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:19.603192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b1fb7-bc14-401f-8fa2-7882aa6f7b47 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:19.647498 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98712879-278b-48de-8380-fccc63529990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:19.648172 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98712879-278b-48de-8380-fccc63529990 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:19.648454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98712879-278b-48de-8380-fccc63529990 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:19.648730 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98712879-278b-48de-8380-fccc63529990 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:19.649814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98712879-278b-48de-8380-fccc63529990 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:19.650125 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98712879-278b-48de-8380-fccc63529990 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:19.651543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98712879-278b-48de-8380-fccc63529990 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:19.664118 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 523/1063] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:18:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:19.670381 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fdc789-e4bd-4c00-aab5-8a045fe79c94 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:19.773616 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fdc789-e4bd-4c00-aab5-8a045fe79c94 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:19.807672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ddc8a-a508-44a8-9e62-11741247d99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:19.808568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ddc8a-a508-44a8-9e62-11741247d99f None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:19.808853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ddc8a-a508-44a8-9e62-11741247d99f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:19.809130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ddc8a-a508-44a8-9e62-11741247d99f None admin] PATH_INFO: `/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/bff6a2d4bc734904b2615608a86b097d/roles/d7b232d8f46040ba8eab9538a784e61f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:19.811359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ddc8a-a508-44a8-9e62-11741247d99f None admin] RBAC: Authorizing `identity:create_grant(project_id=a5f51e47b8e34bf5ac822d8c93853e30, user_id=bff6a2d4bc734904b2615608a86b097d, role_id=d7b232d8f46040ba8eab9538a784e61f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:19.813421 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ddc8a-a508-44a8-9e62-11741247d99f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:19.829239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 541/1064] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:18:19 2022] PUT /identity/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/bff6a2d4bc734904b2615608a86b097d/roles/d7b232d8f46040ba8eab9538a784e61f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:18:20.767764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce3794-d3c4-403f-a741-b6dc770b59ac None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:20.768286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce3794-d3c4-403f-a741-b6dc770b59ac None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:20.768395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce3794-d3c4-403f-a741-b6dc770b59ac None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:20.769889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 524/1065] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:20.775441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241aa79-f0d8-430c-8016-dc0204f8ead6 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:20.775859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241aa79-f0d8-430c-8016-dc0204f8ead6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:20.776001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241aa79-f0d8-430c-8016-dc0204f8ead6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:20.803005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-2241aa79-f0d8-430c-8016-dc0204f8ead6 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:20.804209 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2241aa79-f0d8-430c-8016-dc0204f8ead6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:20.864776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:20.879529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 542/1066] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:20.903437 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce3794-d3c4-403f-a741-b6dc770b59ac None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:20.999761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:21.014529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce3794-d3c4-403f-a741-b6dc770b59ac None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:21.049088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf07de-3392-43db-86dd-f402f636e619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:21.049850 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf07de-3392-43db-86dd-f402f636e619 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:21.050121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf07de-3392-43db-86dd-f402f636e619 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:21.050399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf07de-3392-43db-86dd-f402f636e619 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:21.060959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf07de-3392-43db-86dd-f402f636e619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-2caf07de-3392-43db-86dd-f402f636e619 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:18:21.069226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:21.070230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:21.076550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:21.076550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:21.076550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:21.076550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:21.076550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:18:21.076550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:18:21.076550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:18:21.076550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:18:21.076550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:18:21.076550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:18:21.076550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:18:21.076550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:21.076550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 525/1067] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:18:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:21.078282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241aa79-f0d8-430c-8016-dc0204f8ead6 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:21.177387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241aa79-f0d8-430c-8016-dc0204f8ead6 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:21.211908 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc3a4a-19b3-4e03-be07-f18368fa7fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:21.212576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc3a4a-19b3-4e03-be07-f18368fa7fed None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:21.212859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc3a4a-19b3-4e03-be07-f18368fa7fed None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:21.213155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc3a4a-19b3-4e03-be07-f18368fa7fed None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:21.213905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc3a4a-19b3-4e03-be07-f18368fa7fed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:21.214213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc3a4a-19b3-4e03-be07-f18368fa7fed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:21.215581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc3a4a-19b3-4e03-be07-f18368fa7fed None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:21.228959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 543/1068] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:18:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:21.230477 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf07de-3392-43db-86dd-f402f636e619 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:21.336236 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf07de-3392-43db-86dd-f402f636e619 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:21.369721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d573af-2ad3-41f2-a301-fad659209877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:21.370577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d573af-2ad3-41f2-a301-fad659209877 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:21.370861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d573af-2ad3-41f2-a301-fad659209877 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:21.371137 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d573af-2ad3-41f2-a301-fad659209877 None admin] PATH_INFO: `/v3/users/254364c63eed4d46a6f670fa54431c5c` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:21.384802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d573af-2ad3-41f2-a301-fad659209877 None admin] RBAC: Authorizing `identity:get_user(user_id=254364c63eed4d46a6f670fa54431c5c)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:21.386277 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d573af-2ad3-41f2-a301-fad659209877 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:21.393340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 526/1069] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:18:21 2022] GET /identity/v3/users/254364c63eed4d46a6f670fa54431c5c => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:21.395194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc3a4a-19b3-4e03-be07-f18368fa7fed None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:21.454006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:21.504813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc3a4a-19b3-4e03-be07-f18368fa7fed None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:21.540002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbde464-3616-49f8-9d74-deda45f2beca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:21.540682 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbde464-3616-49f8-9d74-deda45f2beca None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:21.540967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbde464-3616-49f8-9d74-deda45f2beca None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:21.541308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbde464-3616-49f8-9d74-deda45f2beca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:21.549287 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ce7ae-bab5-4f5e-b681-04ae4022eb52 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:21.549566 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ce7ae-bab5-4f5e-b681-04ae4022eb52 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:21.549849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ce7ae-bab5-4f5e-b681-04ae4022eb52 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:21.551360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 527/1070] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:18:21.556186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee4007-7842-49f2-a3be-83a072a3cb71 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:21.556474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee4007-7842-49f2-a3be-83a072a3cb71 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:21.556754 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee4007-7842-49f2-a3be-83a072a3cb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:21.568843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:21.575231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c3ee4007-7842-49f2-a3be-83a072a3cb71 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:21.576460 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c3ee4007-7842-49f2-a3be-83a072a3cb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:21.616421 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbde464-3616-49f8-9d74-deda45f2beca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:21.617902 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbde464-3616-49f8-9d74-deda45f2beca None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:21.653386 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 528/1071] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:18:21.673077 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:21.678165 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee4007-7842-49f2-a3be-83a072a3cb71 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:21.736811 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 544/1072] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:18:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:18:21.738006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbde464-3616-49f8-9d74-deda45f2beca None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:21.783248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee4007-7842-49f2-a3be-83a072a3cb71 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:21.819386 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d447fa4-05ab-4c50-b781-76a194ded049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:21.820067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d447fa4-05ab-4c50-b781-76a194ded049 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:21.820331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d447fa4-05ab-4c50-b781-76a194ded049 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:21.820607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d447fa4-05ab-4c50-b781-76a194ded049 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:21.824389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:21.848948 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbde464-3616-49f8-9d74-deda45f2beca None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:21.885649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362461a0-ccf2-4a19-9b5f-2ef1c216e5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:21.886429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362461a0-ccf2-4a19-9b5f-2ef1c216e5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:21.886707 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362461a0-ccf2-4a19-9b5f-2ef1c216e5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:21.886976 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362461a0-ccf2-4a19-9b5f-2ef1c216e5c1 None admin] PATH_INFO: `/v3/projects/7e43ecd46aa942ce8bb168d683a698d7` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:21.888138 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362461a0-ccf2-4a19-9b5f-2ef1c216e5c1 None admin] RBAC: Authorizing `identity:get_project(project_id=7e43ecd46aa942ce8bb168d683a698d7)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:21.889622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362461a0-ccf2-4a19-9b5f-2ef1c216e5c1 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:21.892061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 545/1073] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:18:21 2022] GET /identity/v3/projects/7e43ecd46aa942ce8bb168d683a698d7 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:21.893246 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665695ec-a538-43b0-a683-2679906684c7 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:21.893531 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665695ec-a538-43b0-a683-2679906684c7 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:21.893635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d447fa4-05ab-4c50-b781-76a194ded049 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:21.893797 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665695ec-a538-43b0-a683-2679906684c7 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:21.895161 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 546/1074] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:21.895267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d447fa4-05ab-4c50-b781-76a194ded049 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:21.899108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665695ec-a538-43b0-a683-2679906684c7 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:21.919857 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:21.994976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 529/1075] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:18:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:18:21.996262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de7927-66ee-4262-92e0-d19d673bacdf None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:21.996551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de7927-66ee-4262-92e0-d19d673bacdf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:21.996829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de7927-66ee-4262-92e0-d19d673bacdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:22.004617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665695ec-a538-43b0-a683-2679906684c7 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:22.012997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-65de7927-66ee-4262-92e0-d19d673bacdf None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:22.014224 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-65de7927-66ee-4262-92e0-d19d673bacdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:22.039524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52273f-51ba-4ce2-b17d-ae2e253e33cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:22.040171 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52273f-51ba-4ce2-b17d-ae2e253e33cb None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:22.040450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52273f-51ba-4ce2-b17d-ae2e253e33cb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:22.040722 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52273f-51ba-4ce2-b17d-ae2e253e33cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:22.041582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52273f-51ba-4ce2-b17d-ae2e253e33cb None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=7e43ecd46aa942ce8bb168d683a698d7, user.id=254364c63eed4d46a6f670fa54431c5c) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:22.041895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52273f-51ba-4ce2-b17d-ae2e253e33cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:22.043320 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52273f-51ba-4ce2-b17d-ae2e253e33cb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:22.044099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:22.049661 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 547/1076] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:18:21 2022] GET /identity/v3/role_assignments?user.id=254364c63eed4d46a6f670fa54431c5c&scope.project.id=7e43ecd46aa942ce8bb168d683a698d7&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:22.050468 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52273f-51ba-4ce2-b17d-ae2e253e33cb None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:22.104781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 530/1077] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:22.130297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de7927-66ee-4262-92e0-d19d673bacdf None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:22.150591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:22.168404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52273f-51ba-4ce2-b17d-ae2e253e33cb None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:22.208077 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb530504-09a1-45f0-abd0-6cf2e6382964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:22.208885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb530504-09a1-45f0-abd0-6cf2e6382964 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:22.209355 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb530504-09a1-45f0-abd0-6cf2e6382964 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:22.209643 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb530504-09a1-45f0-abd0-6cf2e6382964 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:22.223583 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb530504-09a1-45f0-abd0-6cf2e6382964 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:22.225035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb530504-09a1-45f0-abd0-6cf2e6382964 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:22.242844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-cb530504-09a1-45f0-abd0-6cf2e6382964 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:22.242844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:22.242844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:22.242844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:22.242844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:22.242844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:22.242844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:22.242844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:22.242844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:22.242844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:22.242844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:22.242844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:22.242844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:18:22.242844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:18:22.242844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:22.243955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:22.245211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:18:22.245211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:18:22.245211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:22.245211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:22.245211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 548/1078] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:22.250047 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb530504-09a1-45f0-abd0-6cf2e6382964 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:22.258589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de7927-66ee-4262-92e0-d19d673bacdf None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:22.297900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae7790-b370-4aa2-aaeb-f8c7f5076a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:22.298718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae7790-b370-4aa2-aaeb-f8c7f5076a1e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:22.299004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae7790-b370-4aa2-aaeb-f8c7f5076a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:22.299286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae7790-b370-4aa2-aaeb-f8c7f5076a1e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:22.310598 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae7790-b370-4aa2-aaeb-f8c7f5076a1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-5fae7790-b370-4aa2-aaeb-f8c7f5076a1e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:18:22.319952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:22.320888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:22.321765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:22.321765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:22.321765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:22.321765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:22.321765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:18:22.321765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:18:22.321765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:18:22.321765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:18:22.321765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:18:22.321765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:18:22.321765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 15:18:22.321765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:22.321765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 531/1079] 10.222.0.42 () {56 vars in 1120 bytes} [Tue Jul 19 15:18:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:18:22.327141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae7790-b370-4aa2-aaeb-f8c7f5076a1e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:22.355150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:22.361492 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb530504-09a1-45f0-abd0-6cf2e6382964 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:22.401738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32143a5-f6c0-46bb-ac7d-16c0e50ed8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:22.402470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32143a5-f6c0-46bb-ac7d-16c0e50ed8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:22.402757 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32143a5-f6c0-46bb-ac7d-16c0e50ed8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:22.403037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32143a5-f6c0-46bb-ac7d-16c0e50ed8c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:22.403773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32143a5-f6c0-46bb-ac7d-16c0e50ed8c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:22.404086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32143a5-f6c0-46bb-ac7d-16c0e50ed8c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:22.406009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32143a5-f6c0-46bb-ac7d-16c0e50ed8c7 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:22.409230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:22.429170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 549/1080] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:22.443896 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae7790-b370-4aa2-aaeb-f8c7f5076a1e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:22.481949 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8ed9d-64be-4f78-a341-05decab85c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:22.482648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8ed9d-64be-4f78-a341-05decab85c0c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:22.482928 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8ed9d-64be-4f78-a341-05decab85c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:22.483202 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8ed9d-64be-4f78-a341-05decab85c0c None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:22.484538 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8ed9d-64be-4f78-a341-05decab85c0c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:22.484847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8ed9d-64be-4f78-a341-05decab85c0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:22.486272 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8ed9d-64be-4f78-a341-05decab85c0c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:22.500657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 532/1081] 10.222.0.42 () {56 vars in 1115 bytes} [Tue Jul 19 15:18:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:22.502064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32143a5-f6c0-46bb-ac7d-16c0e50ed8c7 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:22.614991 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32143a5-f6c0-46bb-ac7d-16c0e50ed8c7 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:22.651625 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61709bb4-e58c-4ba4-af1e-6da3fac518a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:22.652480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61709bb4-e58c-4ba4-af1e-6da3fac518a2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:22.652766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61709bb4-e58c-4ba4-af1e-6da3fac518a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:22.652983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61709bb4-e58c-4ba4-af1e-6da3fac518a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:22.664647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61709bb4-e58c-4ba4-af1e-6da3fac518a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:22.666179 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61709bb4-e58c-4ba4-af1e-6da3fac518a2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:22.685639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-61709bb4-e58c-4ba4-af1e-6da3fac518a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:22.685639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:22.685639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:22.685639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:22.685639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:22.685639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:22.685639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:22.685639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:22.685639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:22.685639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:22.685639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:22.685639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:22.685639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:18:22.685639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:18:22.685639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:22.686598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:22.687666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:18:22.687666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:18:22.687666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:22.687666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:22.689581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 550/1082] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:18:22.695108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8ed9d-64be-4f78-a341-05decab85c0c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:22.794567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:22.808439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8ed9d-64be-4f78-a341-05decab85c0c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:22.841761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb5935-129f-4a36-9271-2fc72ff9e93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:22.842421 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb5935-129f-4a36-9271-2fc72ff9e93d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:22.842704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb5935-129f-4a36-9271-2fc72ff9e93d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:22.842985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb5935-129f-4a36-9271-2fc72ff9e93d None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:22.844523 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb5935-129f-4a36-9271-2fc72ff9e93d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:22.844830 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb5935-129f-4a36-9271-2fc72ff9e93d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:22.846217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb5935-129f-4a36-9271-2fc72ff9e93d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:22.857531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 533/1083] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:22.863192 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61709bb4-e58c-4ba4-af1e-6da3fac518a2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:22.913665 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:22.963170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61709bb4-e58c-4ba4-af1e-6da3fac518a2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:22.996789 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff269456-2edd-4e50-b830-390e8e703fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:22.997599 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff269456-2edd-4e50-b830-390e8e703fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:22.997881 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff269456-2edd-4e50-b830-390e8e703fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:22.998154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff269456-2edd-4e50-b830-390e8e703fc0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:23.005079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff269456-2edd-4e50-b830-390e8e703fc0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:23.006535 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff269456-2edd-4e50-b830-390e8e703fc0 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-ff269456-2edd-4e50-b830-390e8e703fc0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:18:23.021359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:23.022471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:23.023501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:23.023501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:23.023501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:18:23.023501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:18:23.023501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:18:23.023501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:18:23.023501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:18:23.023501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:18:23.023501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 15:18:23.023501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:23.025289 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 551/1084] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Jul 19 15:18:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:23.028429 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb5935-129f-4a36-9271-2fc72ff9e93d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:23.128306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb5935-129f-4a36-9271-2fc72ff9e93d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:23.164447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e0fd9-28a0-421f-85c2-7fbcbb63c4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:23.165041 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e0fd9-28a0-421f-85c2-7fbcbb63c4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:23.165351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e0fd9-28a0-421f-85c2-7fbcbb63c4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:23.165633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e0fd9-28a0-421f-85c2-7fbcbb63c4b5 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:23.166488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e0fd9-28a0-421f-85c2-7fbcbb63c4b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:23.166803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e0fd9-28a0-421f-85c2-7fbcbb63c4b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:23.168219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e0fd9-28a0-421f-85c2-7fbcbb63c4b5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:23.185564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 534/1085] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Jul 19 15:18:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:23.192287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff269456-2edd-4e50-b830-390e8e703fc0 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:23.293943 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff269456-2edd-4e50-b830-390e8e703fc0 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:23.303090 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:23.328630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472daae0-fb83-47b1-bdd1-79bd746f152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:23.329291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472daae0-fb83-47b1-bdd1-79bd746f152f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:23.329577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472daae0-fb83-47b1-bdd1-79bd746f152f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:23.329853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472daae0-fb83-47b1-bdd1-79bd746f152f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:23.395198 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472daae0-fb83-47b1-bdd1-79bd746f152f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:23.396809 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472daae0-fb83-47b1-bdd1-79bd746f152f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:23.496263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 552/1086] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:18:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:18:23.502967 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e0fd9-28a0-421f-85c2-7fbcbb63c4b5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:23.511790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:23.603644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e0fd9-28a0-421f-85c2-7fbcbb63c4b5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:23.637153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01beb5f-7ace-4563-8ca7-8f354d1b2c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:23.637958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01beb5f-7ace-4563-8ca7-8f354d1b2c98 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:23.638243 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01beb5f-7ace-4563-8ca7-8f354d1b2c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:23.638520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01beb5f-7ace-4563-8ca7-8f354d1b2c98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:23.649331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01beb5f-7ace-4563-8ca7-8f354d1b2c98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:23.650776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01beb5f-7ace-4563-8ca7-8f354d1b2c98 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:23.668381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e01beb5f-7ace-4563-8ca7-8f354d1b2c98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:23.668381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:23.668381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:23.668381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:23.668381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:23.668381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:23.668381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:23.668381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:23.668381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:23.668381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:23.668381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:23.668381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:23.668381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:18:23.668381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:18:23.668381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:23.669470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:23.670431 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:18:23.670431 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:18:23.670431 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:18:23.670431 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:23.670431 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 535/1087] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:23.675646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472daae0-fb83-47b1-bdd1-79bd746f152f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:23.685668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:23.774605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472daae0-fb83-47b1-bdd1-79bd746f152f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:23.807936 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bac2a-3a67-455f-899d-aa38bc3bb1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:23.808594 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bac2a-3a67-455f-899d-aa38bc3bb1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:23.808877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bac2a-3a67-455f-899d-aa38bc3bb1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:23.809175 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bac2a-3a67-455f-899d-aa38bc3bb1a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:23.809905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bac2a-3a67-455f-899d-aa38bc3bb1a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:23.810213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bac2a-3a67-455f-899d-aa38bc3bb1a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:23.811593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bac2a-3a67-455f-899d-aa38bc3bb1a0 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:23.823414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 553/1088] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:23.830416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01beb5f-7ace-4563-8ca7-8f354d1b2c98 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:23.886326 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:23.928276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01beb5f-7ace-4563-8ca7-8f354d1b2c98 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:23.963713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9942b-c5ff-4d28-8b4b-45ef28c66aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:23.964507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9942b-c5ff-4d28-8b4b-45ef28c66aad None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:23.964820 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9942b-c5ff-4d28-8b4b-45ef28c66aad None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:23.965098 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9942b-c5ff-4d28-8b4b-45ef28c66aad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:23.971760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9942b-c5ff-4d28-8b4b-45ef28c66aad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:23.973263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9942b-c5ff-4d28-8b4b-45ef28c66aad None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:23.986857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d2d9942b-c5ff-4d28-8b4b-45ef28c66aad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:18:23.986857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:23.986857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:23.986857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:23.986857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:23.986857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:23.986857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:23.986857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:23.986857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:23.986857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:23.986857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:23.986857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:23.986857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:18:23.986857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:18:23.986857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:23.987911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:23.988954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:18:23.988954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:18:23.988954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:18:23.988954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:18:23.988954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:18:23.988954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:18:23.988954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:18:23.988954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:23.988954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 536/1089] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:18:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:23.993968 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bac2a-3a67-455f-899d-aa38bc3bb1a0 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:24.059802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:24.070722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6c28d-708e-42e1-b459-ff9c1b753398 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:24.071006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6c28d-708e-42e1-b459-ff9c1b753398 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:24.071285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6c28d-708e-42e1-b459-ff9c1b753398 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:24.072656 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 537/1090] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:18:24.078469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c166c1-3660-472e-8fcd-74645e5c0f58 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:24.078740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c166c1-3660-472e-8fcd-74645e5c0f58 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:24.079015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c166c1-3660-472e-8fcd-74645e5c0f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:24.095224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bac2a-3a67-455f-899d-aa38bc3bb1a0 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:24.095411 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-35c166c1-3660-472e-8fcd-74645e5c0f58 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:24.096546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-35c166c1-3660-472e-8fcd-74645e5c0f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:24.129061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97ceae-bb20-4c79-80c3-856dc9342ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:24.129750 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97ceae-bb20-4c79-80c3-856dc9342ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:24.130033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97ceae-bb20-4c79-80c3-856dc9342ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:24.130307 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97ceae-bb20-4c79-80c3-856dc9342ad2 None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:24.131099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97ceae-bb20-4c79-80c3-856dc9342ad2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:24.131405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97ceae-bb20-4c79-80c3-856dc9342ad2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:24.132826 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97ceae-bb20-4c79-80c3-856dc9342ad2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:24.144819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 554/1091] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:18:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:24.151061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97ceae-bb20-4c79-80c3-856dc9342ad2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:24.164947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 538/1092] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:18:24.187296 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c166c1-3660-472e-8fcd-74645e5c0f58 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:24.234668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:24.254778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97ceae-bb20-4c79-80c3-856dc9342ad2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:24.285304 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:24.290269 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c166c1-3660-472e-8fcd-74645e5c0f58 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:24.294198 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba9dbd-c170-482d-9611-237e3355719e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:24.294852 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba9dbd-c170-482d-9611-237e3355719e None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:24.295137 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba9dbd-c170-482d-9611-237e3355719e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:24.295416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba9dbd-c170-482d-9611-237e3355719e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:24.296302 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba9dbd-c170-482d-9611-237e3355719e None admin] RBAC: Adding query filter params (role.id=d7b232d8f46040ba8eab9538a784e61f, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=bff6a2d4bc734904b2615608a86b097d) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:24.296776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba9dbd-c170-482d-9611-237e3355719e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:24.298266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba9dbd-c170-482d-9611-237e3355719e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:24.304204 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 555/1093] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:18:24 2022] GET /identity/v3/role_assignments?user.id=bff6a2d4bc734904b2615608a86b097d&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=d7b232d8f46040ba8eab9538a784e61f => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:18:24.325376 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aaa3ff-72a5-47da-9eaa-e9eeea7fafac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:24.326118 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aaa3ff-72a5-47da-9eaa-e9eeea7fafac None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:24.326402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aaa3ff-72a5-47da-9eaa-e9eeea7fafac None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:24.326677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aaa3ff-72a5-47da-9eaa-e9eeea7fafac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:24.339198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aaa3ff-72a5-47da-9eaa-e9eeea7fafac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e5aaa3ff-72a5-47da-9eaa-e9eeea7fafac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:18:24.347933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:24.349123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:24.350043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:24.350043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:24.350043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:24.350043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:24.350043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:18:24.350043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:18:24.350043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:18:24.350043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:18:24.350043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:18:24.350043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:18:24.350043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:18:24.350043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:24.350043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 539/1094] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:18:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:18:24.355550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba9dbd-c170-482d-9611-237e3355719e None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:24.460166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba9dbd-c170-482d-9611-237e3355719e None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:24.495755 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0251b-573b-49b6-b30b-4e064819f378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:24.496426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0251b-573b-49b6-b30b-4e064819f378 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:24.496708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0251b-573b-49b6-b30b-4e064819f378 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:24.496982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0251b-573b-49b6-b30b-4e064819f378 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:24.497787 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0251b-573b-49b6-b30b-4e064819f378 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:24.498097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0251b-573b-49b6-b30b-4e064819f378 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:24.500040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0251b-573b-49b6-b30b-4e064819f378 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:24.509539 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 556/1095] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:18:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:24.515422 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aaa3ff-72a5-47da-9eaa-e9eeea7fafac None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:24.618933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aaa3ff-72a5-47da-9eaa-e9eeea7fafac None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:24.628559 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:24.654172 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fb00a-a901-44f2-8bac-b5e2dff30a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:24.655037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fb00a-a901-44f2-8bac-b5e2dff30a1c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:24.655322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fb00a-a901-44f2-8bac-b5e2dff30a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:24.656169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fb00a-a901-44f2-8bac-b5e2dff30a1c None admin] PATH_INFO: `/v3/users/254364c63eed4d46a6f670fa54431c5c` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:24.657405 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fb00a-a901-44f2-8bac-b5e2dff30a1c None admin] RBAC: Authorizing `identity:get_user(user_id=254364c63eed4d46a6f670fa54431c5c)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:24.658847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fb00a-a901-44f2-8bac-b5e2dff30a1c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:24.660488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 540/1096] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:18:24 2022] GET /identity/v3/users/254364c63eed4d46a6f670fa54431c5c => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:24.666758 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0251b-573b-49b6-b30b-4e064819f378 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:24.726598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:24.772135 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0251b-573b-49b6-b30b-4e064819f378 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:24.809339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b41fbc-a21a-463e-96b8-b94478a7e63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:24.810211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b41fbc-a21a-463e-96b8-b94478a7e63a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:24.810495 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b41fbc-a21a-463e-96b8-b94478a7e63a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:24.810768 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b41fbc-a21a-463e-96b8-b94478a7e63a None admin] PATH_INFO: `/v3/projects/7e43ecd46aa942ce8bb168d683a698d7` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:24.811980 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b41fbc-a21a-463e-96b8-b94478a7e63a None admin] RBAC: Authorizing `identity:get_project(project_id=7e43ecd46aa942ce8bb168d683a698d7)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:24.813517 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b41fbc-a21a-463e-96b8-b94478a7e63a None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:24.815231 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 557/1097] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:18:24 2022] GET /identity/v3/projects/7e43ecd46aa942ce8bb168d683a698d7 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:24.821468 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fb00a-a901-44f2-8bac-b5e2dff30a1c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:24.927084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fb00a-a901-44f2-8bac-b5e2dff30a1c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:24.963948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850c430-c865-42a1-ab9a-1a203e76cae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:24.964858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850c430-c865-42a1-ab9a-1a203e76cae0 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:24.965171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850c430-c865-42a1-ab9a-1a203e76cae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:24.965596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850c430-c865-42a1-ab9a-1a203e76cae0 None admin] PATH_INFO: `/v3/projects/7e43ecd46aa942ce8bb168d683a698d7/users/254364c63eed4d46a6f670fa54431c5c/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:24.967550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850c430-c865-42a1-ab9a-1a203e76cae0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e43ecd46aa942ce8bb168d683a698d7, user_id=254364c63eed4d46a6f670fa54431c5c, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:24.969053 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850c430-c865-42a1-ab9a-1a203e76cae0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:24.981997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 541/1098] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:18:24 2022] PUT /identity/v3/projects/7e43ecd46aa942ce8bb168d683a698d7/users/254364c63eed4d46a6f670fa54431c5c/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:18:26.987890 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15672e-d128-4965-813f-dc4eebcafacd None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:26.988427 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15672e-d128-4965-813f-dc4eebcafacd None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:26.988427 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15672e-d128-4965-813f-dc4eebcafacd None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:26.989832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 558/1099] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:26.995133 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc82c9f-6901-459e-8038-22f8254065e4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:26.995505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc82c9f-6901-459e-8038-22f8254065e4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:26.995697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc82c9f-6901-459e-8038-22f8254065e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:27.012673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8dc82c9f-6901-459e-8038-22f8254065e4 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:27.013894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8dc82c9f-6901-459e-8038-22f8254065e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:27.042137 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:27.081269 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 542/1100] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:27.104547 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15672e-d128-4965-813f-dc4eebcafacd None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:27.208249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15672e-d128-4965-813f-dc4eebcafacd None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:27.230236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:27.241842 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2626d-8ead-4e8e-b3aa-d1f2c5bfe60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:27.242563 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2626d-8ead-4e8e-b3aa-d1f2c5bfe60a None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:27.242851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2626d-8ead-4e8e-b3aa-d1f2c5bfe60a None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:27.243125 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2626d-8ead-4e8e-b3aa-d1f2c5bfe60a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:27.253305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2626d-8ead-4e8e-b3aa-d1f2c5bfe60a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-e7d2626d-8ead-4e8e-b3aa-d1f2c5bfe60a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:18:27.260820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:27.261942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:27.262970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:27.262970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:27.262970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:27.262970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:27.262970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:18:27.262970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:18:27.262970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:18:27.262970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:18:27.262970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:18:27.262970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:18:27.262970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:18:27.262970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:27.262970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 559/1101] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:18:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:27.268807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc82c9f-6901-459e-8038-22f8254065e4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:27.364615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc82c9f-6901-459e-8038-22f8254065e4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:27.397265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeca2dd-7beb-41bc-9670-9a49a8567340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:27.397920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeca2dd-7beb-41bc-9670-9a49a8567340 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:27.398204 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeca2dd-7beb-41bc-9670-9a49a8567340 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:27.398485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeca2dd-7beb-41bc-9670-9a49a8567340 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:27.399498 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeca2dd-7beb-41bc-9670-9a49a8567340 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:27.399808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeca2dd-7beb-41bc-9670-9a49a8567340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:27.401184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeca2dd-7beb-41bc-9670-9a49a8567340 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:27.409274 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 543/1102] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:18:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:27.415098 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2626d-8ead-4e8e-b3aa-d1f2c5bfe60a None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:27.513415 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2626d-8ead-4e8e-b3aa-d1f2c5bfe60a None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:27.547973 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2a476-908b-4a23-b705-16b10a7ac82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:27.548830 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2a476-908b-4a23-b705-16b10a7ac82c None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:27.549116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2a476-908b-4a23-b705-16b10a7ac82c None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:27.549427 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2a476-908b-4a23-b705-16b10a7ac82c None admin] PATH_INFO: `/v3/users/254364c63eed4d46a6f670fa54431c5c` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:27.550594 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff2a476-908b-4a23-b705-16b10a7ac82c None admin] RBAC: Authorizing `identity:get_user(user_id=254364c63eed4d46a6f670fa54431c5c)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:27.552032 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff2a476-908b-4a23-b705-16b10a7ac82c None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:27.553673 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 560/1103] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:18:27 2022] GET /identity/v3/users/254364c63eed4d46a6f670fa54431c5c => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:27.559339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeca2dd-7beb-41bc-9670-9a49a8567340 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:27.587893 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:27.661048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeca2dd-7beb-41bc-9670-9a49a8567340 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:27.694465 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a039f-99d5-4d9f-9abd-12421e4a56d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:27.695105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a039f-99d5-4d9f-9abd-12421e4a56d7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:27.695391 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a039f-99d5-4d9f-9abd-12421e4a56d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:27.695668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a039f-99d5-4d9f-9abd-12421e4a56d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:27.761305 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a039f-99d5-4d9f-9abd-12421e4a56d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:27.762736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a039f-99d5-4d9f-9abd-12421e4a56d7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:27.789085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:27.857978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 544/1104] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:18:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:18:27.864621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2a476-908b-4a23-b705-16b10a7ac82c None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:27.914053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:27.965020 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2a476-908b-4a23-b705-16b10a7ac82c None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:27.997697 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb7888-ae37-4f44-b28a-86505e9f28d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:27.998545 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb7888-ae37-4f44-b28a-86505e9f28d5 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:27.998826 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb7888-ae37-4f44-b28a-86505e9f28d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:27.999283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb7888-ae37-4f44-b28a-86505e9f28d5 None admin] PATH_INFO: `/v3/projects/7e43ecd46aa942ce8bb168d683a698d7` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:28.000463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb7888-ae37-4f44-b28a-86505e9f28d5 None admin] RBAC: Authorizing `identity:get_project(project_id=7e43ecd46aa942ce8bb168d683a698d7)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:28.001925 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb7888-ae37-4f44-b28a-86505e9f28d5 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:28.003606 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 561/1105] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:18:27 2022] GET /identity/v3/projects/7e43ecd46aa942ce8bb168d683a698d7 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:28.009792 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a039f-99d5-4d9f-9abd-12421e4a56d7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:28.105350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a039f-99d5-4d9f-9abd-12421e4a56d7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:28.133172 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:28.141846 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce46a04-c5eb-4066-80a7-cd0c70d4174f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:28.142486 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce46a04-c5eb-4066-80a7-cd0c70d4174f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:28.142764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce46a04-c5eb-4066-80a7-cd0c70d4174f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:28.143038 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce46a04-c5eb-4066-80a7-cd0c70d4174f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:28.143861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce46a04-c5eb-4066-80a7-cd0c70d4174f None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=7e43ecd46aa942ce8bb168d683a698d7, user.id=254364c63eed4d46a6f670fa54431c5c) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:28.144168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce46a04-c5eb-4066-80a7-cd0c70d4174f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:28.145617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce46a04-c5eb-4066-80a7-cd0c70d4174f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:28.151529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 545/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:18:28 2022] GET /identity/v3/role_assignments?user.id=254364c63eed4d46a6f670fa54431c5c&scope.project.id=7e43ecd46aa942ce8bb168d683a698d7&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:30.187930 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da12cc4-508c-487b-a486-538dc9702ce4 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:30.188582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da12cc4-508c-487b-a486-538dc9702ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:30.188582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da12cc4-508c-487b-a486-538dc9702ce4 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:30.189889 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 562/1107] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:30.195302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e994b-bf3e-4255-8e01-babce467e234 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:30.195730 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e994b-bf3e-4255-8e01-babce467e234 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:30.195870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e994b-bf3e-4255-8e01-babce467e234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:30.213107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ce7e994b-bf3e-4255-8e01-babce467e234 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:30.214320 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ce7e994b-bf3e-4255-8e01-babce467e234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:30.281246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 546/1108] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:30.305966 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da12cc4-508c-487b-a486-538dc9702ce4 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:30.405589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da12cc4-508c-487b-a486-538dc9702ce4 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:30.422580 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:30.439776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305366b-4dfc-43f5-83a7-b6556324fb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:30.440501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305366b-4dfc-43f5-83a7-b6556324fb09 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:30.440782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305366b-4dfc-43f5-83a7-b6556324fb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:30.441057 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305366b-4dfc-43f5-83a7-b6556324fb09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:30.442255 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305366b-4dfc-43f5-83a7-b6556324fb09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:30.443721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305366b-4dfc-43f5-83a7-b6556324fb09 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:30.445366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 563/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:30.451248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e994b-bf3e-4255-8e01-babce467e234 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:30.548059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e994b-bf3e-4255-8e01-babce467e234 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:30.570210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:30.581723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d15def-3bed-4512-9540-b4b388e151d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:30.582394 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d15def-3bed-4512-9540-b4b388e151d9 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:30.582682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d15def-3bed-4512-9540-b4b388e151d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:30.582966 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d15def-3bed-4512-9540-b4b388e151d9 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:30.583674 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d15def-3bed-4512-9540-b4b388e151d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:30.585104 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d15def-3bed-4512-9540-b4b388e151d9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:30.604132 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 547/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:18:30 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:18:32.645023 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6275e-f122-43db-83da-f11931163323 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:32.645565 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6275e-f122-43db-83da-f11931163323 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:32.645565 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6275e-f122-43db-83da-f11931163323 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:32.646905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 564/1111] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:32.652212 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec07767-910e-4fa7-a419-87692f5fbc71 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:32.652643 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec07767-910e-4fa7-a419-87692f5fbc71 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:32.652773 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec07767-910e-4fa7-a419-87692f5fbc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:32.669317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-aec07767-910e-4fa7-a419-87692f5fbc71 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:32.671105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-aec07767-910e-4fa7-a419-87692f5fbc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:32.735546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 548/1112] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:32.758598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6275e-f122-43db-83da-f11931163323 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:32.858868 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6275e-f122-43db-83da-f11931163323 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:32.893334 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa76c7a-0069-4490-94f2-7dc77f3450bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:32.894191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa76c7a-0069-4490-94f2-7dc77f3450bc None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:32.894477 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa76c7a-0069-4490-94f2-7dc77f3450bc None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:32.894758 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa76c7a-0069-4490-94f2-7dc77f3450bc None admin] PATH_INFO: `/v3/roles/44bd561cc5594ffd9bcc27ef6dad0fc5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:32.896293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa76c7a-0069-4490-94f2-7dc77f3450bc None admin] RBAC: Authorizing `identity:get_role(role_id=44bd561cc5594ffd9bcc27ef6dad0fc5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:32.897664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa76c7a-0069-4490-94f2-7dc77f3450bc None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:32.899312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 565/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:18:32 2022] GET /identity/v3/roles/44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:32.905210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec07767-910e-4fa7-a419-87692f5fbc71 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:33.009503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec07767-910e-4fa7-a419-87692f5fbc71 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:33.019161 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:33.044805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838f0b4-47f6-4782-9ac6-366045af5475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:33.046172 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838f0b4-47f6-4782-9ac6-366045af5475 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:33.046454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838f0b4-47f6-4782-9ac6-366045af5475 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:33.046730 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838f0b4-47f6-4782-9ac6-366045af5475 None admin] PATH_INFO: `/v3/users/06d53e1d8e4245ebbb8de066fcf82487` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:33.059387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838f0b4-47f6-4782-9ac6-366045af5475 None admin] RBAC: Authorizing `identity:get_user(user_id=06d53e1d8e4245ebbb8de066fcf82487)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:33.060822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838f0b4-47f6-4782-9ac6-366045af5475 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:33.063466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 549/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:18:32 2022] GET /identity/v3/users/06d53e1d8e4245ebbb8de066fcf82487 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:33.069212 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa76c7a-0069-4490-94f2-7dc77f3450bc None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:33.122132 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:33.172035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa76c7a-0069-4490-94f2-7dc77f3450bc None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:33.213741 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48524995-e85d-4eef-b1a4-bc1c69eb19c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:33.214392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48524995-e85d-4eef-b1a4-bc1c69eb19c0 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:33.214675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48524995-e85d-4eef-b1a4-bc1c69eb19c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:33.214955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48524995-e85d-4eef-b1a4-bc1c69eb19c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:33.297900 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48524995-e85d-4eef-b1a4-bc1c69eb19c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:33.299353 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48524995-e85d-4eef-b1a4-bc1c69eb19c0 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:33.352048 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:33.409623 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 566/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:18:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:18:33.415678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838f0b4-47f6-4782-9ac6-366045af5475 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:33.512022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838f0b4-47f6-4782-9ac6-366045af5475 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:33.552458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b815c70-81d2-424d-90c5-82b2f14c7c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:33.553352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b815c70-81d2-424d-90c5-82b2f14c7c27 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:33.553633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b815c70-81d2-424d-90c5-82b2f14c7c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:33.553908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b815c70-81d2-424d-90c5-82b2f14c7c27 None admin] PATH_INFO: `/v3/projects/7e43ecd46aa942ce8bb168d683a698d7` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:33.555089 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b815c70-81d2-424d-90c5-82b2f14c7c27 None admin] RBAC: Authorizing `identity:get_project(project_id=7e43ecd46aa942ce8bb168d683a698d7)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:33.556520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b815c70-81d2-424d-90c5-82b2f14c7c27 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:33.558269 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 550/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:18:33 2022] GET /identity/v3/projects/7e43ecd46aa942ce8bb168d683a698d7 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:33.565507 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48524995-e85d-4eef-b1a4-bc1c69eb19c0 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:33.679679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48524995-e85d-4eef-b1a4-bc1c69eb19c0 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:33.716992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee39f9-cd09-4e62-b014-bdd67bf47a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:33.717717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee39f9-cd09-4e62-b014-bdd67bf47a07 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:33.718008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee39f9-cd09-4e62-b014-bdd67bf47a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:33.718291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee39f9-cd09-4e62-b014-bdd67bf47a07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:33.719440 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee39f9-cd09-4e62-b014-bdd67bf47a07 None admin] RBAC: Adding query filter params (role.id=44bd561cc5594ffd9bcc27ef6dad0fc5, scope.project.id=7e43ecd46aa942ce8bb168d683a698d7, user.id=06d53e1d8e4245ebbb8de066fcf82487) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:33.719760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee39f9-cd09-4e62-b014-bdd67bf47a07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:33.721240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee39f9-cd09-4e62-b014-bdd67bf47a07 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:33.726699 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 567/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:18:33 2022] GET /identity/v3/role_assignments?user.id=06d53e1d8e4245ebbb8de066fcf82487&scope.project.id=7e43ecd46aa942ce8bb168d683a698d7&role.id=44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:35.748988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994feab-7ac6-40df-b427-45fc59dddbed None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:35.749532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994feab-7ac6-40df-b427-45fc59dddbed None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:35.749654 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994feab-7ac6-40df-b427-45fc59dddbed None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:35.750977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 551/1118] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:35.756649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12083fbf-681b-4c3f-9c74-57a507c5bd20 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:35.757041 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12083fbf-681b-4c3f-9c74-57a507c5bd20 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:35.757247 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12083fbf-681b-4c3f-9c74-57a507c5bd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:35.773841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-12083fbf-681b-4c3f-9c74-57a507c5bd20 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:35.775026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-12083fbf-681b-4c3f-9c74-57a507c5bd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:35.788390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:35.841645 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 568/1119] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:35.864791 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994feab-7ac6-40df-b427-45fc59dddbed None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:35.899214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:35.964630 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994feab-7ac6-40df-b427-45fc59dddbed None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:35.997829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c688f-3ba4-48ac-bf4d-d6a3ed089f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:35.998687 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c688f-3ba4-48ac-bf4d-d6a3ed089f2b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:35.998951 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c688f-3ba4-48ac-bf4d-d6a3ed089f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:35.999272 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c688f-3ba4-48ac-bf4d-d6a3ed089f2b None admin] PATH_INFO: `/v3/roles/44bd561cc5594ffd9bcc27ef6dad0fc5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:36.000676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c688f-3ba4-48ac-bf4d-d6a3ed089f2b None admin] RBAC: Authorizing `identity:get_role(role_id=44bd561cc5594ffd9bcc27ef6dad0fc5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:36.002058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c688f-3ba4-48ac-bf4d-d6a3ed089f2b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:36.003412 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 552/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:18:35 2022] GET /identity/v3/roles/44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:36.009309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12083fbf-681b-4c3f-9c74-57a507c5bd20 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:36.106057 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12083fbf-681b-4c3f-9c74-57a507c5bd20 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:36.139452 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183651e-d2c9-476d-ae1f-27b9f0eeab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:36.140382 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183651e-d2c9-476d-ae1f-27b9f0eeab78 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:36.140671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183651e-d2c9-476d-ae1f-27b9f0eeab78 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:36.140951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183651e-d2c9-476d-ae1f-27b9f0eeab78 None admin] PATH_INFO: `/v3/users/06d53e1d8e4245ebbb8de066fcf82487` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:36.142288 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183651e-d2c9-476d-ae1f-27b9f0eeab78 None admin] RBAC: Authorizing `identity:get_user(user_id=06d53e1d8e4245ebbb8de066fcf82487)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:36.143723 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183651e-d2c9-476d-ae1f-27b9f0eeab78 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:36.145374 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 569/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:18:36 2022] GET /identity/v3/users/06d53e1d8e4245ebbb8de066fcf82487 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:36.151177 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c688f-3ba4-48ac-bf4d-d6a3ed089f2b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:36.248987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c688f-3ba4-48ac-bf4d-d6a3ed089f2b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:36.251101 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:36.283979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e596bd-6723-482b-8c81-ec0ac9d767e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:36.284834 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e596bd-6723-482b-8c81-ec0ac9d767e9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:36.285115 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e596bd-6723-482b-8c81-ec0ac9d767e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:36.285428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e596bd-6723-482b-8c81-ec0ac9d767e9 None admin] PATH_INFO: `/v3/projects/7e43ecd46aa942ce8bb168d683a698d7` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:36.286593 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e596bd-6723-482b-8c81-ec0ac9d767e9 None admin] RBAC: Authorizing `identity:get_project(project_id=7e43ecd46aa942ce8bb168d683a698d7)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:36.288017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e596bd-6723-482b-8c81-ec0ac9d767e9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:36.289747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 553/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:18:36 2022] GET /identity/v3/projects/7e43ecd46aa942ce8bb168d683a698d7 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:36.296459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183651e-d2c9-476d-ae1f-27b9f0eeab78 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:36.314979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:36.395681 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183651e-d2c9-476d-ae1f-27b9f0eeab78 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:36.429337 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274780c-82b8-4896-b116-e1c3a93f7a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:36.430213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274780c-82b8-4896-b116-e1c3a93f7a65 None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:36.430493 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274780c-82b8-4896-b116-e1c3a93f7a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:36.430766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274780c-82b8-4896-b116-e1c3a93f7a65 None admin] PATH_INFO: `/v3/projects/7e43ecd46aa942ce8bb168d683a698d7/users/06d53e1d8e4245ebbb8de066fcf82487/roles/44bd561cc5594ffd9bcc27ef6dad0fc5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:36.432945 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274780c-82b8-4896-b116-e1c3a93f7a65 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e43ecd46aa942ce8bb168d683a698d7, user_id=06d53e1d8e4245ebbb8de066fcf82487, role_id=44bd561cc5594ffd9bcc27ef6dad0fc5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:36.434427 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274780c-82b8-4896-b116-e1c3a93f7a65 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:36.447245 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 570/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:18:36 2022] PUT /identity/v3/projects/7e43ecd46aa942ce8bb168d683a698d7/users/06d53e1d8e4245ebbb8de066fcf82487/roles/44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:18:38.475238 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98849f8-d5e2-4192-ad48-07c272e74b0a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:38.475806 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98849f8-d5e2-4192-ad48-07c272e74b0a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:38.475806 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98849f8-d5e2-4192-ad48-07c272e74b0a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:38.477197 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 554/1124] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:38.482373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e2d7e-ed79-4ab5-af3f-f9850e2cc288 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:38.483507 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e2d7e-ed79-4ab5-af3f-f9850e2cc288 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:38.483616 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e2d7e-ed79-4ab5-af3f-f9850e2cc288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:38.500620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-a01e2d7e-ed79-4ab5-af3f-f9850e2cc288 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:38.501835 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a01e2d7e-ed79-4ab5-af3f-f9850e2cc288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:38.567079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 571/1125] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:38.590528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98849f8-d5e2-4192-ad48-07c272e74b0a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:38.691284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98849f8-d5e2-4192-ad48-07c272e74b0a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:38.724303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181ced4-602d-4562-b4b1-31957ebb4afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:38.725164 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181ced4-602d-4562-b4b1-31957ebb4afb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:38.725463 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181ced4-602d-4562-b4b1-31957ebb4afb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:38.725749 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181ced4-602d-4562-b4b1-31957ebb4afb None admin] PATH_INFO: `/v3/roles/44bd561cc5594ffd9bcc27ef6dad0fc5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:38.727214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181ced4-602d-4562-b4b1-31957ebb4afb None admin] RBAC: Authorizing `identity:get_role(role_id=44bd561cc5594ffd9bcc27ef6dad0fc5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:38.728614 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181ced4-602d-4562-b4b1-31957ebb4afb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:38.730012 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 555/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:18:38 2022] GET /identity/v3/roles/44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:38.736411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e2d7e-ed79-4ab5-af3f-f9850e2cc288 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:38.744476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:38.834426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e2d7e-ed79-4ab5-af3f-f9850e2cc288 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:38.867900 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae9027-ae63-422c-b196-d5d5765320b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:38.868763 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae9027-ae63-422c-b196-d5d5765320b2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:38.869049 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae9027-ae63-422c-b196-d5d5765320b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:38.869356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae9027-ae63-422c-b196-d5d5765320b2 None admin] PATH_INFO: `/v3/users/06d53e1d8e4245ebbb8de066fcf82487` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:38.870534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae9027-ae63-422c-b196-d5d5765320b2 None admin] RBAC: Authorizing `identity:get_user(user_id=06d53e1d8e4245ebbb8de066fcf82487)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:38.871960 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae9027-ae63-422c-b196-d5d5765320b2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:38.873660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 572/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:18:38 2022] GET /identity/v3/users/06d53e1d8e4245ebbb8de066fcf82487 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:38.879456 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181ced4-602d-4562-b4b1-31957ebb4afb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:38.919695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:38.982514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181ced4-602d-4562-b4b1-31957ebb4afb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:39.017794 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26ab36-c06d-4863-8cac-90145b907f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:39.018432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26ab36-c06d-4863-8cac-90145b907f10 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:39.018717 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26ab36-c06d-4863-8cac-90145b907f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:39.018997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26ab36-c06d-4863-8cac-90145b907f10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:39.085531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26ab36-c06d-4863-8cac-90145b907f10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:39.086918 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26ab36-c06d-4863-8cac-90145b907f10 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:39.124359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:39.183777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 556/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:18:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 15:18:39.189968 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae9027-ae63-422c-b196-d5d5765320b2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:39.263232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:39.290490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae9027-ae63-422c-b196-d5d5765320b2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:39.323935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4cdb5-2b13-449b-81a5-0c2b968466a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:39.324785 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4cdb5-2b13-449b-81a5-0c2b968466a1 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:39.325069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4cdb5-2b13-449b-81a5-0c2b968466a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:39.325371 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4cdb5-2b13-449b-81a5-0c2b968466a1 None admin] PATH_INFO: `/v3/projects/7e43ecd46aa942ce8bb168d683a698d7` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:39.327456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4cdb5-2b13-449b-81a5-0c2b968466a1 None admin] RBAC: Authorizing `identity:get_project(project_id=7e43ecd46aa942ce8bb168d683a698d7)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:39.328907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4cdb5-2b13-449b-81a5-0c2b968466a1 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:39.330638 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 573/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:18:39 2022] GET /identity/v3/projects/7e43ecd46aa942ce8bb168d683a698d7 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:39.336973 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26ab36-c06d-4863-8cac-90145b907f10 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:39.434177 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26ab36-c06d-4863-8cac-90145b907f10 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:39.466133 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff4e402-c747-4076-a25f-e4ca2d596dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:39.466770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4e402-c747-4076-a25f-e4ca2d596dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:39.467656 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4e402-c747-4076-a25f-e4ca2d596dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:39.467888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4e402-c747-4076-a25f-e4ca2d596dc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:39.468709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff4e402-c747-4076-a25f-e4ca2d596dc6 None admin] RBAC: Adding query filter params (role.id=44bd561cc5594ffd9bcc27ef6dad0fc5, scope.project.id=7e43ecd46aa942ce8bb168d683a698d7, user.id=06d53e1d8e4245ebbb8de066fcf82487) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:39.469016 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff4e402-c747-4076-a25f-e4ca2d596dc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:39.470452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff4e402-c747-4076-a25f-e4ca2d596dc6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:39.476034 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 557/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:18:39 2022] GET /identity/v3/role_assignments?user.id=06d53e1d8e4245ebbb8de066fcf82487&scope.project.id=7e43ecd46aa942ce8bb168d683a698d7&role.id=44bd561cc5594ffd9bcc27ef6dad0fc5 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:41.491844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7baad0-48a3-4abd-987c-2a3671e7f0b6 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:41.492369 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7baad0-48a3-4abd-987c-2a3671e7f0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:41.492460 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7baad0-48a3-4abd-987c-2a3671e7f0b6 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:41.493784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 574/1131] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:41.499023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e568009-cb99-48df-9082-ad72a30977f6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:41.499420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e568009-cb99-48df-9082-ad72a30977f6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:41.499584 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e568009-cb99-48df-9082-ad72a30977f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:41.504901 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:41.518370 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0e568009-cb99-48df-9082-ad72a30977f6 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:41.519551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0e568009-cb99-48df-9082-ad72a30977f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:41.585601 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 558/1132] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:18:41.609449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7baad0-48a3-4abd-987c-2a3671e7f0b6 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:41.709904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7baad0-48a3-4abd-987c-2a3671e7f0b6 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:41.742224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e31f90-e2ea-4a5c-b685-d1e41f87dda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:41.742999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e31f90-e2ea-4a5c-b685-d1e41f87dda9 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:41.743278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e31f90-e2ea-4a5c-b685-d1e41f87dda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:41.743551 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e31f90-e2ea-4a5c-b685-d1e41f87dda9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:41.744708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e31f90-e2ea-4a5c-b685-d1e41f87dda9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:41.746164 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e31f90-e2ea-4a5c-b685-d1e41f87dda9 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:41.747763 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 575/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:41.754821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e568009-cb99-48df-9082-ad72a30977f6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:41.851705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e568009-cb99-48df-9082-ad72a30977f6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:41.880473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:41.885724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52590c4f-b307-42b4-9975-d9577a4864c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:41.886379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52590c4f-b307-42b4-9975-d9577a4864c2 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:41.886662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52590c4f-b307-42b4-9975-d9577a4864c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:41.886936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52590c4f-b307-42b4-9975-d9577a4864c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:41.887648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52590c4f-b307-42b4-9975-d9577a4864c2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:41.889098 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52590c4f-b307-42b4-9975-d9577a4864c2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:41.914866 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 559/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Jul 19 15:18:41 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:18:44.006847 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14498d47-29a1-4310-94d4-edbf02063ccd None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:44.007375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14498d47-29a1-4310-94d4-edbf02063ccd None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:44.007375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14498d47-29a1-4310-94d4-edbf02063ccd None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:44.008765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 576/1135] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:44.014591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee90a2d-fac7-4437-832f-8e2fb8cfc8cb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:44.015025 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee90a2d-fac7-4437-832f-8e2fb8cfc8cb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:44.015130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee90a2d-fac7-4437-832f-8e2fb8cfc8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:44.032282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-fee90a2d-fac7-4437-832f-8e2fb8cfc8cb None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:44.033480 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fee90a2d-fac7-4437-832f-8e2fb8cfc8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:44.097918 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 560/1136] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:44.121945 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14498d47-29a1-4310-94d4-edbf02063ccd None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:44.134667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:44.222381 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14498d47-29a1-4310-94d4-edbf02063ccd None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:44.255726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1c563-9dc4-4447-aa77-836a45f08d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:44.256515 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c563-9dc4-4447-aa77-836a45f08d0e None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:44.256794 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c563-9dc4-4447-aa77-836a45f08d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:44.257113 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c563-9dc4-4447-aa77-836a45f08d0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:44.258293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1c563-9dc4-4447-aa77-836a45f08d0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:44.259721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1c563-9dc4-4447-aa77-836a45f08d0e None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:44.261355 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 577/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:18:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:44.267074 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee90a2d-fac7-4437-832f-8e2fb8cfc8cb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:44.364548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee90a2d-fac7-4437-832f-8e2fb8cfc8cb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:44.378283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:44.399375 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e20ba-ec0c-4373-b880-9e097ffa66f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:44.400044 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e20ba-ec0c-4373-b880-9e097ffa66f1 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:44.400328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e20ba-ec0c-4373-b880-9e097ffa66f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:44.400606 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e20ba-ec0c-4373-b880-9e097ffa66f1 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:44.401347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e20ba-ec0c-4373-b880-9e097ffa66f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:44.402768 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e20ba-ec0c-4373-b880-9e097ffa66f1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:44.423168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 561/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:18:44 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:18:46.483363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ddb75-10dd-4a1f-a2c1-90ff66bc2fc8 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:46.483883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ddb75-10dd-4a1f-a2c1-90ff66bc2fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:46.483883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ddb75-10dd-4a1f-a2c1-90ff66bc2fc8 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:46.485293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 578/1139] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:46.490871 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09055d-c961-46d8-90e0-d18de037fc52 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:46.491286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09055d-c961-46d8-90e0-d18de037fc52 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:46.491437 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09055d-c961-46d8-90e0-d18de037fc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:46.508090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8b09055d-c961-46d8-90e0-d18de037fc52 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:46.509257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8b09055d-c961-46d8-90e0-d18de037fc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:46.574075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 562/1140] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:46.597400 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ddb75-10dd-4a1f-a2c1-90ff66bc2fc8 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:46.673997 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:46.695848 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ddb75-10dd-4a1f-a2c1-90ff66bc2fc8 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:46.729163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e74c3b-7fba-4b73-8e1c-ad9c5bc59163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:46.729953 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e74c3b-7fba-4b73-8e1c-ad9c5bc59163 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:46.730241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e74c3b-7fba-4b73-8e1c-ad9c5bc59163 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:46.730516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e74c3b-7fba-4b73-8e1c-ad9c5bc59163 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:46.740285 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e74c3b-7fba-4b73-8e1c-ad9c5bc59163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-e6e74c3b-7fba-4b73-8e1c-ad9c5bc59163 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:18:46.748026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:46.749235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:46.750154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:46.750154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:46.750154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:46.750154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:46.750154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:18:46.750154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:18:46.750154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:18:46.750154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:18:46.750154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:18:46.750154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:18:46.750154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:18:46.750154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:46.750154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 579/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:18:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:46.755188 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09055d-c961-46d8-90e0-d18de037fc52 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:46.849491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:46.852449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09055d-c961-46d8-90e0-d18de037fc52 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:46.886063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0c294-3926-4c1b-91e8-ab7f98ca3c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:46.886723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0c294-3926-4c1b-91e8-ab7f98ca3c34 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:46.887010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0c294-3926-4c1b-91e8-ab7f98ca3c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:46.887288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0c294-3926-4c1b-91e8-ab7f98ca3c34 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:46.888037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0c294-3926-4c1b-91e8-ab7f98ca3c34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:46.888879 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0c294-3926-4c1b-91e8-ab7f98ca3c34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:46.890275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0c294-3926-4c1b-91e8-ab7f98ca3c34 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:46.898581 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 563/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:18:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:46.905094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e74c3b-7fba-4b73-8e1c-ad9c5bc59163 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:47.001566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e74c3b-7fba-4b73-8e1c-ad9c5bc59163 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:47.040528 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27c81d-cdc5-43ca-838b-050290eff91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:47.041415 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27c81d-cdc5-43ca-838b-050290eff91b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:47.041698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27c81d-cdc5-43ca-838b-050290eff91b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:47.041970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27c81d-cdc5-43ca-838b-050290eff91b None admin] PATH_INFO: `/v3/users/5164026649924f27992303db33b21afb` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:47.049795 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:47.055701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27c81d-cdc5-43ca-838b-050290eff91b None admin] RBAC: Authorizing `identity:get_user(user_id=5164026649924f27992303db33b21afb)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:47.057149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27c81d-cdc5-43ca-838b-050290eff91b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:47.058800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 580/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:18:46 2022] GET /identity/v3/users/5164026649924f27992303db33b21afb => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:47.065187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0c294-3926-4c1b-91e8-ab7f98ca3c34 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:47.174144 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0c294-3926-4c1b-91e8-ab7f98ca3c34 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:47.207927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94442d2-b389-4b96-b78c-da01ce97c71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:47.208572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94442d2-b389-4b96-b78c-da01ce97c71a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:47.208857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94442d2-b389-4b96-b78c-da01ce97c71a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:47.209183 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94442d2-b389-4b96-b78c-da01ce97c71a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:47.238316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:47.277112 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94442d2-b389-4b96-b78c-da01ce97c71a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:47.278649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94442d2-b389-4b96-b78c-da01ce97c71a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:47.372020 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 564/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:18:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:18:47.378372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27c81d-cdc5-43ca-838b-050290eff91b None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:47.475834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27c81d-cdc5-43ca-838b-050290eff91b None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:47.508988 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bfb7d-f118-449a-a209-f5dbaf8189cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:47.509866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bfb7d-f118-449a-a209-f5dbaf8189cc None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:47.510333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bfb7d-f118-449a-a209-f5dbaf8189cc None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:47.510612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bfb7d-f118-449a-a209-f5dbaf8189cc None admin] PATH_INFO: `/v3/projects/72aec519212c40f9908df0822e4e0526` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:47.511767 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bfb7d-f118-449a-a209-f5dbaf8189cc None admin] RBAC: Authorizing `identity:get_project(project_id=72aec519212c40f9908df0822e4e0526)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:47.513246 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bfb7d-f118-449a-a209-f5dbaf8189cc None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:47.515155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 581/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:18:47 2022] GET /identity/v3/projects/72aec519212c40f9908df0822e4e0526 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:47.521316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94442d2-b389-4b96-b78c-da01ce97c71a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:47.581954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:47.619308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94442d2-b389-4b96-b78c-da01ce97c71a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:47.652925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc85113-9301-4cbb-9823-1208829261aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:47.653601 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc85113-9301-4cbb-9823-1208829261aa None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:47.654515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc85113-9301-4cbb-9823-1208829261aa None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:47.654798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc85113-9301-4cbb-9823-1208829261aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:47.655616 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc85113-9301-4cbb-9823-1208829261aa None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=72aec519212c40f9908df0822e4e0526, user.id=5164026649924f27992303db33b21afb) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:47.655926 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc85113-9301-4cbb-9823-1208829261aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:47.657365 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc85113-9301-4cbb-9823-1208829261aa None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:47.662615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 565/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:18:47 2022] GET /identity/v3/role_assignments?user.id=5164026649924f27992303db33b21afb&scope.project.id=72aec519212c40f9908df0822e4e0526&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:49.661751 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62daa703-dfa9-407f-8e08-db6528abd893 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:49.662272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62daa703-dfa9-407f-8e08-db6528abd893 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:49.662272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62daa703-dfa9-407f-8e08-db6528abd893 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:49.663563 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 582/1147] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:49.668889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96aa196-d915-4177-9ab8-5f95da9dbd49 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:49.669371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96aa196-d915-4177-9ab8-5f95da9dbd49 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:49.670138 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96aa196-d915-4177-9ab8-5f95da9dbd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:49.686677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c96aa196-d915-4177-9ab8-5f95da9dbd49 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:49.687854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c96aa196-d915-4177-9ab8-5f95da9dbd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:49.752385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 566/1148] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:49.774973 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62daa703-dfa9-407f-8e08-db6528abd893 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:49.837465 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:49.876175 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62daa703-dfa9-407f-8e08-db6528abd893 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:49.910359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6de27-1000-4efc-b702-c0673045cd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:49.911144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6de27-1000-4efc-b702-c0673045cd31 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:49.911430 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6de27-1000-4efc-b702-c0673045cd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:49.911709 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6de27-1000-4efc-b702-c0673045cd31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:49.922235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6de27-1000-4efc-b702-c0673045cd31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-15a6de27-1000-4efc-b702-c0673045cd31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:18:49.929816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:49.930894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:49.931935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:49.931935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:49.931935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:49.931935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:49.931935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:18:49.931935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:18:49.931935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:18:49.931935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:18:49.931935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:18:49.931935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:18:49.931935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:18:49.931935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:49.931935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 583/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:18:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:49.937256 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96aa196-d915-4177-9ab8-5f95da9dbd49 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:49.991144 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:50.033890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96aa196-d915-4177-9ab8-5f95da9dbd49 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:50.066609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a4293-6f35-4ee4-9a53-ac0791d2a8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:50.067271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a4293-6f35-4ee4-9a53-ac0791d2a8bd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:50.067555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a4293-6f35-4ee4-9a53-ac0791d2a8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:50.067832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a4293-6f35-4ee4-9a53-ac0791d2a8bd None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:50.068577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a4293-6f35-4ee4-9a53-ac0791d2a8bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:50.068887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a4293-6f35-4ee4-9a53-ac0791d2a8bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:50.070272 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a4293-6f35-4ee4-9a53-ac0791d2a8bd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:50.078794 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 567/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:18:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:50.084521 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6de27-1000-4efc-b702-c0673045cd31 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:50.180748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6de27-1000-4efc-b702-c0673045cd31 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:50.202877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:50.213934 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ff6a0-9d2a-4cfb-84b3-fb7b49cb1065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:50.214784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ff6a0-9d2a-4cfb-84b3-fb7b49cb1065 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:50.215064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ff6a0-9d2a-4cfb-84b3-fb7b49cb1065 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:50.215337 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ff6a0-9d2a-4cfb-84b3-fb7b49cb1065 None admin] PATH_INFO: `/v3/users/5164026649924f27992303db33b21afb` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:50.217021 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8ff6a0-9d2a-4cfb-84b3-fb7b49cb1065 None admin] RBAC: Authorizing `identity:get_user(user_id=5164026649924f27992303db33b21afb)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:50.218470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8ff6a0-9d2a-4cfb-84b3-fb7b49cb1065 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:50.220052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 584/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:18:50 2022] GET /identity/v3/users/5164026649924f27992303db33b21afb => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:50.225960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a4293-6f35-4ee4-9a53-ac0791d2a8bd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:50.323535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a4293-6f35-4ee4-9a53-ac0791d2a8bd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:50.339568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:50.356834 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c10c64-fc06-4d1c-ade9-06622f2e5eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:50.358208 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c10c64-fc06-4d1c-ade9-06622f2e5eaa None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:50.358485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c10c64-fc06-4d1c-ade9-06622f2e5eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:50.358758 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c10c64-fc06-4d1c-ade9-06622f2e5eaa None admin] PATH_INFO: `/v3/projects/72aec519212c40f9908df0822e4e0526` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:50.359923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c10c64-fc06-4d1c-ade9-06622f2e5eaa None admin] RBAC: Authorizing `identity:get_project(project_id=72aec519212c40f9908df0822e4e0526)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:50.361369 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c10c64-fc06-4d1c-ade9-06622f2e5eaa None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:50.363061 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 568/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:18:50 2022] GET /identity/v3/projects/72aec519212c40f9908df0822e4e0526 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:50.368981 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ff6a0-9d2a-4cfb-84b3-fb7b49cb1065 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:50.466858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ff6a0-9d2a-4cfb-84b3-fb7b49cb1065 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:50.498739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356f20f-e23f-44cd-8e55-f3eea68a01fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:50.499637 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356f20f-e23f-44cd-8e55-f3eea68a01fb None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:50.499920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356f20f-e23f-44cd-8e55-f3eea68a01fb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:50.500203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356f20f-e23f-44cd-8e55-f3eea68a01fb None admin] PATH_INFO: `/v3/projects/72aec519212c40f9908df0822e4e0526/users/5164026649924f27992303db33b21afb/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:50.502194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356f20f-e23f-44cd-8e55-f3eea68a01fb None admin] RBAC: Authorizing `identity:create_grant(project_id=72aec519212c40f9908df0822e4e0526, user_id=5164026649924f27992303db33b21afb, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:50.503617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356f20f-e23f-44cd-8e55-f3eea68a01fb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:50.517000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 585/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:18:50 2022] PUT /identity/v3/projects/72aec519212c40f9908df0822e4e0526/users/5164026649924f27992303db33b21afb/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:18:52.509209 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47369b01-d85e-4cd8-a33f-1c71e95370d9 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:52.509702 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47369b01-d85e-4cd8-a33f-1c71e95370d9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:52.509702 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47369b01-d85e-4cd8-a33f-1c71e95370d9 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:52.511242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 569/1154] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:52.517257 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac6d18-6e87-402d-95a5-f2ed2f6559b5 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:52.517680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac6d18-6e87-402d-95a5-f2ed2f6559b5 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:52.517815 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac6d18-6e87-402d-95a5-f2ed2f6559b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:52.535871 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-7aac6d18-6e87-402d-95a5-f2ed2f6559b5 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:52.537083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7aac6d18-6e87-402d-95a5-f2ed2f6559b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:52.583726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:52.604712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 586/1155] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:18:52.628593 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47369b01-d85e-4cd8-a33f-1c71e95370d9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:52.729258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47369b01-d85e-4cd8-a33f-1c71e95370d9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:52.761740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3e1b1-10c8-49a9-abf4-1a46feca5a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:52.762462 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3e1b1-10c8-49a9-abf4-1a46feca5a11 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:52.762744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3e1b1-10c8-49a9-abf4-1a46feca5a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:52.763710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3e1b1-10c8-49a9-abf4-1a46feca5a11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:52.774059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3e1b1-10c8-49a9-abf4-1a46feca5a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-b8f3e1b1-10c8-49a9-abf4-1a46feca5a11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:18:52.782403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:52.783509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:52.784710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:52.784710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:52.784710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:52.784710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:52.784710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:18:52.784710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:18:52.784710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:18:52.784710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:18:52.784710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:18:52.784710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:18:52.784710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:18:52.784710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:18:52.784710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 570/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:18:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:52.789619 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac6d18-6e87-402d-95a5-f2ed2f6559b5 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:52.883933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac6d18-6e87-402d-95a5-f2ed2f6559b5 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:52.917684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca1410-ddec-4d19-9e6e-6026fbbb0593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:52.918342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca1410-ddec-4d19-9e6e-6026fbbb0593 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:52.918626 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca1410-ddec-4d19-9e6e-6026fbbb0593 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:52.918901 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca1410-ddec-4d19-9e6e-6026fbbb0593 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:52.919671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca1410-ddec-4d19-9e6e-6026fbbb0593 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:52.919979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca1410-ddec-4d19-9e6e-6026fbbb0593 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:52.921368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca1410-ddec-4d19-9e6e-6026fbbb0593 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:52.929155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 587/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:18:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:52.934785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3e1b1-10c8-49a9-abf4-1a46feca5a11 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:52.981714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:53.034151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3e1b1-10c8-49a9-abf4-1a46feca5a11 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:53.066523 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806883f5-9608-4e74-94d9-c2ddf37e06d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:53.067378 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806883f5-9608-4e74-94d9-c2ddf37e06d6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:53.067661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806883f5-9608-4e74-94d9-c2ddf37e06d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:53.067936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806883f5-9608-4e74-94d9-c2ddf37e06d6 None admin] PATH_INFO: `/v3/users/5164026649924f27992303db33b21afb` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:53.069293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806883f5-9608-4e74-94d9-c2ddf37e06d6 None admin] RBAC: Authorizing `identity:get_user(user_id=5164026649924f27992303db33b21afb)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:53.070813 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806883f5-9608-4e74-94d9-c2ddf37e06d6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:53.072489 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 571/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:18:52 2022] GET /identity/v3/users/5164026649924f27992303db33b21afb => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:53.079482 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca1410-ddec-4d19-9e6e-6026fbbb0593 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:53.125381 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:53.180232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca1410-ddec-4d19-9e6e-6026fbbb0593 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:53.212950 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8e113-80db-4acc-b0ac-1f1b49601ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:53.213616 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e113-80db-4acc-b0ac-1f1b49601ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:53.213904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e113-80db-4acc-b0ac-1f1b49601ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:53.214182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e113-80db-4acc-b0ac-1f1b49601ef2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:53.285866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8e113-80db-4acc-b0ac-1f1b49601ef2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:53.287266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8e113-80db-4acc-b0ac-1f1b49601ef2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:53.329973 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:53.383821 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 588/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:18:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:18:53.389936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806883f5-9608-4e74-94d9-c2ddf37e06d6 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:53.488377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806883f5-9608-4e74-94d9-c2ddf37e06d6 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:53.501996 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:53.524912 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a6891-266e-493d-9c29-bdc737b5f41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:53.525822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a6891-266e-493d-9c29-bdc737b5f41d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:53.526107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a6891-266e-493d-9c29-bdc737b5f41d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:53.526387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a6891-266e-493d-9c29-bdc737b5f41d None admin] PATH_INFO: `/v3/projects/72aec519212c40f9908df0822e4e0526` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:53.527567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a6891-266e-493d-9c29-bdc737b5f41d None admin] RBAC: Authorizing `identity:get_project(project_id=72aec519212c40f9908df0822e4e0526)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:53.530209 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a6891-266e-493d-9c29-bdc737b5f41d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:53.531932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 572/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:18:53 2022] GET /identity/v3/projects/72aec519212c40f9908df0822e4e0526 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:53.538070 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8e113-80db-4acc-b0ac-1f1b49601ef2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:53.634354 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8e113-80db-4acc-b0ac-1f1b49601ef2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:53.668477 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9b22d-8065-42c7-9348-eb90400840fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:53.669312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9b22d-8065-42c7-9348-eb90400840fb None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:53.669691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9b22d-8065-42c7-9348-eb90400840fb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:53.669979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9b22d-8065-42c7-9348-eb90400840fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:53.670815 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9b22d-8065-42c7-9348-eb90400840fb None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=72aec519212c40f9908df0822e4e0526, user.id=5164026649924f27992303db33b21afb) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:53.671134 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9b22d-8065-42c7-9348-eb90400840fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:53.672567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9b22d-8065-42c7-9348-eb90400840fb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:53.678248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 589/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:18:53 2022] GET /identity/v3/role_assignments?user.id=5164026649924f27992303db33b21afb&scope.project.id=72aec519212c40f9908df0822e4e0526&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:55.716178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813bc71-46d9-47e6-b323-2efb42833b40 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:55.716702 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813bc71-46d9-47e6-b323-2efb42833b40 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:55.716702 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813bc71-46d9-47e6-b323-2efb42833b40 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:55.718043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 573/1162] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:55.723142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2f5d8-bfb6-4125-84ad-0bdf2658b000 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:55.723566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2f5d8-bfb6-4125-84ad-0bdf2658b000 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:55.723701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2f5d8-bfb6-4125-84ad-0bdf2658b000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:55.740714 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-7ff2f5d8-bfb6-4125-84ad-0bdf2658b000 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:55.741925 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7ff2f5d8-bfb6-4125-84ad-0bdf2658b000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:55.743862 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:55.808066 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 590/1163] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:18:55.831309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813bc71-46d9-47e6-b323-2efb42833b40 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:55.936058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813bc71-46d9-47e6-b323-2efb42833b40 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:55.970556 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57ca6d-fd72-444e-86cd-41fca68dc67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:55.971146 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57ca6d-fd72-444e-86cd-41fca68dc67f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:55.971431 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57ca6d-fd72-444e-86cd-41fca68dc67f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:55.971712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57ca6d-fd72-444e-86cd-41fca68dc67f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:56.022773 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:56.042274 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f57ca6d-fd72-444e-86cd-41fca68dc67f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:56.043754 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f57ca6d-fd72-444e-86cd-41fca68dc67f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:56.143462 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 574/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:18:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:18:56.149696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2f5d8-bfb6-4125-84ad-0bdf2658b000 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:56.246067 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2f5d8-bfb6-4125-84ad-0bdf2658b000 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:56.279680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e94d26-74cb-405e-81d0-755bb6d6f2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:56.280472 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e94d26-74cb-405e-81d0-755bb6d6f2ce None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:56.280756 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e94d26-74cb-405e-81d0-755bb6d6f2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:56.281028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e94d26-74cb-405e-81d0-755bb6d6f2ce None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:56.287549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e94d26-74cb-405e-81d0-755bb6d6f2ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:56.288992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e94d26-74cb-405e-81d0-755bb6d6f2ce None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:56.290652 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-c2e94d26-74cb-405e-81d0-755bb6d6f2ce None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:18:56.304678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:18:56.305938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:18:56.306817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:18:56.306817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:18:56.306817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:18:56.306817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:18:56.306817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:18:56.306817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:18:56.306817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:18:56.306817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:18:56.306817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:18:56.306817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:18:56.306817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 19 15:18:56.306817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:18:56.306817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 19 15:18:56.306817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:18:56.306817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 591/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Jul 19 15:18:56 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:18:56.311877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57ca6d-fd72-444e-86cd-41fca68dc67f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:56.390885 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:18:56.409247 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57ca6d-fd72-444e-86cd-41fca68dc67f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:56.441789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cf1a9-3100-4a22-9fbf-5f7dd1de0d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:56.442441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cf1a9-3100-4a22-9fbf-5f7dd1de0d00 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:56.442722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cf1a9-3100-4a22-9fbf-5f7dd1de0d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:56.442997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cf1a9-3100-4a22-9fbf-5f7dd1de0d00 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:56.443713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cf1a9-3100-4a22-9fbf-5f7dd1de0d00 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:18:56.444019 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cf1a9-3100-4a22-9fbf-5f7dd1de0d00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:56.445424 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cf1a9-3100-4a22-9fbf-5f7dd1de0d00 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:56.452876 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 575/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Tue Jul 19 15:18:56 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:18:58.454961 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164b812-02d0-41f5-ad1e-b8a425a27844 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:58.455485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164b812-02d0-41f5-ad1e-b8a425a27844 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:58.455485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164b812-02d0-41f5-ad1e-b8a425a27844 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:58.456875 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 592/1167] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:18:58.462113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f518e-7666-4899-8916-572ce8904bb5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:58.462481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f518e-7666-4899-8916-572ce8904bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:58.462668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f518e-7666-4899-8916-572ce8904bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:58.479862 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d90f518e-7666-4899-8916-572ce8904bb5 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:18:58.481028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d90f518e-7666-4899-8916-572ce8904bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:18:58.545910 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 576/1168] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:18:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:18:58.567890 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164b812-02d0-41f5-ad1e-b8a425a27844 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:18:58.666100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164b812-02d0-41f5-ad1e-b8a425a27844 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:18:58.698527 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8527ed-8468-4cc3-aeb4-7acbb0ce2aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:18:58.699159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8527ed-8468-4cc3-aeb4-7acbb0ce2aad None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:18:58.699442 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8527ed-8468-4cc3-aeb4-7acbb0ce2aad None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:18:58.699717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8527ed-8468-4cc3-aeb4-7acbb0ce2aad None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:18:58.701027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8527ed-8468-4cc3-aeb4-7acbb0ce2aad None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:18:58.702418 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8527ed-8468-4cc3-aeb4-7acbb0ce2aad None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:18:58.717170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 593/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Tue Jul 19 15:18:58 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:19:00.773944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8f307-77b7-47e1-b835-11c82e0037db None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:00.774515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8f307-77b7-47e1-b835-11c82e0037db None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:00.774515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8f307-77b7-47e1-b835-11c82e0037db None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:00.775877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 577/1170] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:19:00.781759 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348a4c6-dbcf-48f8-9ad8-f8c71e7e93bf None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:00.782201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348a4c6-dbcf-48f8-9ad8-f8c71e7e93bf None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:00.782325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348a4c6-dbcf-48f8-9ad8-f8c71e7e93bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:00.799266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-9348a4c6-dbcf-48f8-9ad8-f8c71e7e93bf None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:19:00.800447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9348a4c6-dbcf-48f8-9ad8-f8c71e7e93bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:19:00.851650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:00.866388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 594/1171] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:19:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:19:00.889403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8f307-77b7-47e1-b835-11c82e0037db None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:00.971503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:00.994779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8f307-77b7-47e1-b835-11c82e0037db None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:01.028969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58593c11-69af-4abc-b0ed-3d310e11ac81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:01.029847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58593c11-69af-4abc-b0ed-3d310e11ac81 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:01.030132 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58593c11-69af-4abc-b0ed-3d310e11ac81 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:01.030411 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58593c11-69af-4abc-b0ed-3d310e11ac81 None admin] PATH_INFO: `/v3/domains/95409186b72441fab7147bd35433a667` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:01.040482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58593c11-69af-4abc-b0ed-3d310e11ac81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=95409186b72441fab7147bd35433a667)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:01.041968 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58593c11-69af-4abc-b0ed-3d310e11ac81 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:01.043621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 578/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Tue Jul 19 15:19:00 2022] GET /identity/v3/domains/95409186b72441fab7147bd35433a667 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:19:01.049596 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348a4c6-dbcf-48f8-9ad8-f8c71e7e93bf None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:01.147591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348a4c6-dbcf-48f8-9ad8-f8c71e7e93bf None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:01.181876 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f346649-2805-4725-ae21-ae9a6db6252f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:01.182533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f346649-2805-4725-ae21-ae9a6db6252f None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:01.182821 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f346649-2805-4725-ae21-ae9a6db6252f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:01.183103 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f346649-2805-4725-ae21-ae9a6db6252f None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:01.183807 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f346649-2805-4725-ae21-ae9a6db6252f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:01.186139 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f346649-2805-4725-ae21-ae9a6db6252f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:01.210453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 595/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Jul 19 15:19:01 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:19:03.247011 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8513a3-251d-491a-8ba3-d66a0abbdd98 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:03.247555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8513a3-251d-491a-8ba3-d66a0abbdd98 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:03.247654 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8513a3-251d-491a-8ba3-d66a0abbdd98 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:03.249098 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 579/1174] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:19:03.254394 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb139f-68b7-4f60-ac6c-9d9c8be209c3 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:03.254802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb139f-68b7-4f60-ac6c-9d9c8be209c3 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:03.254956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb139f-68b7-4f60-ac6c-9d9c8be209c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:03.271864 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-27cb139f-68b7-4f60-ac6c-9d9c8be209c3 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:19:03.273349 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-27cb139f-68b7-4f60-ac6c-9d9c8be209c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:19:03.306247 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:03.339053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 596/1175] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:19:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:19:03.364406 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8513a3-251d-491a-8ba3-d66a0abbdd98 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:03.465269 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8513a3-251d-491a-8ba3-d66a0abbdd98 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:03.499431 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4206168-8854-4607-8090-c29649915fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:03.500221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4206168-8854-4607-8090-c29649915fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:03.500508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4206168-8854-4607-8090-c29649915fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:03.500789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4206168-8854-4607-8090-c29649915fb1 None admin] PATH_INFO: `/v3/domains/95409186b72441fab7147bd35433a667` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:03.502009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4206168-8854-4607-8090-c29649915fb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=95409186b72441fab7147bd35433a667)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:03.503440 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4206168-8854-4607-8090-c29649915fb1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:03.505048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 580/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Tue Jul 19 15:19:03 2022] GET /identity/v3/domains/95409186b72441fab7147bd35433a667 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:19:03.510619 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb139f-68b7-4f60-ac6c-9d9c8be209c3 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:03.607395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb139f-68b7-4f60-ac6c-9d9c8be209c3 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:03.640129 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b4b01-b3f9-418d-a1f1-2061d5a93d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:03.640786 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b4b01-b3f9-418d-a1f1-2061d5a93d05 None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:03.641072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b4b01-b3f9-418d-a1f1-2061d5a93d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:03.641379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b4b01-b3f9-418d-a1f1-2061d5a93d05 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:03.642077 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b4b01-b3f9-418d-a1f1-2061d5a93d05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:03.643512 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b4b01-b3f9-418d-a1f1-2061d5a93d05 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:03.664019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 597/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:19:03 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:19:05.710876 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b6421-24ec-4d34-a96f-a3345da6ba9c None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:05.711389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b6421-24ec-4d34-a96f-a3345da6ba9c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:05.711389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b6421-24ec-4d34-a96f-a3345da6ba9c None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:05.712716 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 581/1178] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:19:05.718580 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6164b99-624e-4d55-817d-9bad67b3d48a None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:05.718957 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6164b99-624e-4d55-817d-9bad67b3d48a None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:05.719135 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6164b99-624e-4d55-817d-9bad67b3d48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:05.735468 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-f6164b99-624e-4d55-817d-9bad67b3d48a None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:19:05.736649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f6164b99-624e-4d55-817d-9bad67b3d48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:19:05.756115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:05.802791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 598/1179] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:19:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:19:05.826730 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b6421-24ec-4d34-a96f-a3345da6ba9c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:05.830477 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:05.924738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b6421-24ec-4d34-a96f-a3345da6ba9c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:05.957852 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa43ecb-2ea0-44ba-9d22-2712a8a338b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:05.958631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa43ecb-2ea0-44ba-9d22-2712a8a338b2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:05.958913 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa43ecb-2ea0-44ba-9d22-2712a8a338b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:05.959187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa43ecb-2ea0-44ba-9d22-2712a8a338b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:05.969768 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa43ecb-2ea0-44ba-9d22-2712a8a338b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-7fa43ecb-2ea0-44ba-9d22-2712a8a338b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:19:05.977317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:19:05.978515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:19:05.979639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:19:05.979639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:19:05.979639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:19:05.979639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:19:05.979639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:19:05.979639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:19:05.979639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:19:05.979639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:19:05.979639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:19:05.979639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:19:05.979639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:19:05.979639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:19:05.979639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 582/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:19:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:19:05.985078 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6164b99-624e-4d55-817d-9bad67b3d48a None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:06.083559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6164b99-624e-4d55-817d-9bad67b3d48a None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:06.116538 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171832bd-f2ab-4bd3-bfb4-4c7e557854f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:06.117224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171832bd-f2ab-4bd3-bfb4-4c7e557854f4 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:06.117508 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171832bd-f2ab-4bd3-bfb4-4c7e557854f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:06.117784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171832bd-f2ab-4bd3-bfb4-4c7e557854f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:06.118531 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171832bd-f2ab-4bd3-bfb4-4c7e557854f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:19:06.118838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171832bd-f2ab-4bd3-bfb4-4c7e557854f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:06.120213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171832bd-f2ab-4bd3-bfb4-4c7e557854f4 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:06.129002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 599/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:19:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:19:06.134689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa43ecb-2ea0-44ba-9d22-2712a8a338b2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:06.195586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:06.232147 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa43ecb-2ea0-44ba-9d22-2712a8a338b2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:06.265341 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9806ba-2925-47ea-b081-c3254a5c553b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:06.266209 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9806ba-2925-47ea-b081-c3254a5c553b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:06.266492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9806ba-2925-47ea-b081-c3254a5c553b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:06.266768 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9806ba-2925-47ea-b081-c3254a5c553b None admin] PATH_INFO: `/v3/users/8a3f8666aad34e24af32e78099ac4b57` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:06.279890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9806ba-2925-47ea-b081-c3254a5c553b None admin] RBAC: Authorizing `identity:get_user(user_id=8a3f8666aad34e24af32e78099ac4b57)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:06.281357 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9806ba-2925-47ea-b081-c3254a5c553b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:06.283001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 583/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:19:06 2022] GET /identity/v3/users/8a3f8666aad34e24af32e78099ac4b57 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:19:06.288862 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171832bd-f2ab-4bd3-bfb4-4c7e557854f4 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:06.308486 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:06.389842 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171832bd-f2ab-4bd3-bfb4-4c7e557854f4 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:06.423466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bd04b-966d-41af-acfa-d0ecd286a141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:06.424100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bd04b-966d-41af-acfa-d0ecd286a141 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:06.424386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bd04b-966d-41af-acfa-d0ecd286a141 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:06.424665 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bd04b-966d-41af-acfa-d0ecd286a141 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:06.490763 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499bd04b-966d-41af-acfa-d0ecd286a141 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:06.492142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499bd04b-966d-41af-acfa-d0ecd286a141 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:06.509919 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:06.590551 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 600/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:19:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:19:06.596596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9806ba-2925-47ea-b081-c3254a5c553b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:06.692584 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9806ba-2925-47ea-b081-c3254a5c553b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:06.714625 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:06.725980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f6159-056c-454d-8f8c-148726781b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:06.726829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f6159-056c-454d-8f8c-148726781b5d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:06.727108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f6159-056c-454d-8f8c-148726781b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:06.727381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f6159-056c-454d-8f8c-148726781b5d None admin] PATH_INFO: `/v3/projects/35c20d8fc8974f2faa4416f6ac927943` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:06.728545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f6159-056c-454d-8f8c-148726781b5d None admin] RBAC: Authorizing `identity:get_project(project_id=35c20d8fc8974f2faa4416f6ac927943)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:06.729990 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f6159-056c-454d-8f8c-148726781b5d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:06.731704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 584/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:19:06 2022] GET /identity/v3/projects/35c20d8fc8974f2faa4416f6ac927943 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:19:06.737858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bd04b-966d-41af-acfa-d0ecd286a141 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:06.835554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bd04b-966d-41af-acfa-d0ecd286a141 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:06.851802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:06.868110 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa090b7-a08b-4e7f-a37f-f5088c92ba4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:06.868747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa090b7-a08b-4e7f-a37f-f5088c92ba4f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:06.869028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa090b7-a08b-4e7f-a37f-f5088c92ba4f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:06.869327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa090b7-a08b-4e7f-a37f-f5088c92ba4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:06.870147 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa090b7-a08b-4e7f-a37f-f5088c92ba4f None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=35c20d8fc8974f2faa4416f6ac927943, user.id=8a3f8666aad34e24af32e78099ac4b57) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:19:06.870454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa090b7-a08b-4e7f-a37f-f5088c92ba4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:06.872575 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa090b7-a08b-4e7f-a37f-f5088c92ba4f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:06.877841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 601/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:19:06 2022] GET /identity/v3/role_assignments?user.id=8a3f8666aad34e24af32e78099ac4b57&scope.project.id=35c20d8fc8974f2faa4416f6ac927943&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:19:08.876355 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b85bf-2544-4eec-9096-7f5436caee93 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:08.876892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b85bf-2544-4eec-9096-7f5436caee93 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:08.876892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b85bf-2544-4eec-9096-7f5436caee93 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:08.878327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 585/1186] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:19:08.883556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3538f-58b7-419c-90b9-33e56bc684b3 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:08.883968 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3538f-58b7-419c-90b9-33e56bc684b3 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:08.884115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3538f-58b7-419c-90b9-33e56bc684b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:08.901973 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-84a3538f-58b7-419c-90b9-33e56bc684b3 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:19:08.903565 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-84a3538f-58b7-419c-90b9-33e56bc684b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:19:08.969935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 602/1187] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:19:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:19:08.992764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b85bf-2544-4eec-9096-7f5436caee93 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:09.093462 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b85bf-2544-4eec-9096-7f5436caee93 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:09.126959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca5886-58c6-4e65-8768-7e362a3b5d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:09.127677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca5886-58c6-4e65-8768-7e362a3b5d2d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:09.127961 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca5886-58c6-4e65-8768-7e362a3b5d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:09.128235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca5886-58c6-4e65-8768-7e362a3b5d2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:09.138468 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca5886-58c6-4e65-8768-7e362a3b5d2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-59ca5886-58c6-4e65-8768-7e362a3b5d2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:19:09.146023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:19:09.147393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:19:09.148432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:19:09.148432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:19:09.148432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:19:09.148432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:19:09.148432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:19:09.148432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:19:09.148432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:19:09.148432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:19:09.148432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:19:09.148432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:19:09.148432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:19:09.148432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:19:09.148432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 586/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:19:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:19:09.155973 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3538f-58b7-419c-90b9-33e56bc684b3 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:09.254656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3538f-58b7-419c-90b9-33e56bc684b3 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:09.267957 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:09.288924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b5c47-0586-4860-9043-9a4408205bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:09.289624 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b5c47-0586-4860-9043-9a4408205bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:09.289909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b5c47-0586-4860-9043-9a4408205bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:09.290187 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b5c47-0586-4860-9043-9a4408205bb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:09.290933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b5c47-0586-4860-9043-9a4408205bb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:19:09.291241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b5c47-0586-4860-9043-9a4408205bb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:09.292603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b5c47-0586-4860-9043-9a4408205bb7 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:09.300982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 603/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:19:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:19:09.306984 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca5886-58c6-4e65-8768-7e362a3b5d2d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:09.356370 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:09.406233 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca5886-58c6-4e65-8768-7e362a3b5d2d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:09.438896 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd856a-7ede-487f-8225-a83c1cf4c68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:09.439746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd856a-7ede-487f-8225-a83c1cf4c68a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:09.440027 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd856a-7ede-487f-8225-a83c1cf4c68a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:09.440303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd856a-7ede-487f-8225-a83c1cf4c68a None admin] PATH_INFO: `/v3/users/8a3f8666aad34e24af32e78099ac4b57` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:09.441485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd856a-7ede-487f-8225-a83c1cf4c68a None admin] RBAC: Authorizing `identity:get_user(user_id=8a3f8666aad34e24af32e78099ac4b57)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:09.442898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd856a-7ede-487f-8225-a83c1cf4c68a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:09.444498 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 587/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:19:09 2022] GET /identity/v3/users/8a3f8666aad34e24af32e78099ac4b57 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:19:09.450534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b5c47-0586-4860-9043-9a4408205bb7 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:09.548317 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b5c47-0586-4860-9043-9a4408205bb7 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:09.580234 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29837f40-ffc5-4f50-912b-fe1af6d2abaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:09.581074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29837f40-ffc5-4f50-912b-fe1af6d2abaf None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:09.581435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29837f40-ffc5-4f50-912b-fe1af6d2abaf None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:09.581709 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29837f40-ffc5-4f50-912b-fe1af6d2abaf None admin] PATH_INFO: `/v3/projects/35c20d8fc8974f2faa4416f6ac927943` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:09.582880 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29837f40-ffc5-4f50-912b-fe1af6d2abaf None admin] RBAC: Authorizing `identity:get_project(project_id=35c20d8fc8974f2faa4416f6ac927943)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:09.584306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29837f40-ffc5-4f50-912b-fe1af6d2abaf None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:09.586075 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 604/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:19:09 2022] GET /identity/v3/projects/35c20d8fc8974f2faa4416f6ac927943 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:19:09.592994 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd856a-7ede-487f-8225-a83c1cf4c68a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:09.690919 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd856a-7ede-487f-8225-a83c1cf4c68a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:09.706959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:09.723954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe6948-6288-4250-8f0a-4f59de3b3aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:09.724834 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe6948-6288-4250-8f0a-4f59de3b3aa3 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:09.725121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe6948-6288-4250-8f0a-4f59de3b3aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:09.725429 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe6948-6288-4250-8f0a-4f59de3b3aa3 None admin] PATH_INFO: `/v3/projects/35c20d8fc8974f2faa4416f6ac927943/users/8a3f8666aad34e24af32e78099ac4b57/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:09.727395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe6948-6288-4250-8f0a-4f59de3b3aa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=35c20d8fc8974f2faa4416f6ac927943, user_id=8a3f8666aad34e24af32e78099ac4b57, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:09.728819 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe6948-6288-4250-8f0a-4f59de3b3aa3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:09.742114 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 588/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:19:09 2022] PUT /identity/v3/projects/35c20d8fc8974f2faa4416f6ac927943/users/8a3f8666aad34e24af32e78099ac4b57/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:19:11.718568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328019a-4185-46e9-a0cf-4309f17a0196 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:11.719080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328019a-4185-46e9-a0cf-4309f17a0196 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:11.719080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328019a-4185-46e9-a0cf-4309f17a0196 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:11.720413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 605/1193] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:19:11.725612 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfcae2-0860-4067-885f-e8ec30cdca61 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:11.725991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfcae2-0860-4067-885f-e8ec30cdca61 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:11.726165 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfcae2-0860-4067-885f-e8ec30cdca61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:11.743021 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-70bfcae2-0860-4067-885f-e8ec30cdca61 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:19:11.744190 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-70bfcae2-0860-4067-885f-e8ec30cdca61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:19:11.808515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 589/1194] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:19:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:19:11.831252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328019a-4185-46e9-a0cf-4309f17a0196 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:11.875000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:11.931489 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328019a-4185-46e9-a0cf-4309f17a0196 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:11.967081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe93eb-7400-460c-bd26-8bd774c5fed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:11.967858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe93eb-7400-460c-bd26-8bd774c5fed2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:11.968138 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe93eb-7400-460c-bd26-8bd774c5fed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:11.968411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe93eb-7400-460c-bd26-8bd774c5fed2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:11.978174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe93eb-7400-460c-bd26-8bd774c5fed2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-25fe93eb-7400-460c-bd26-8bd774c5fed2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:19:11.986326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:19:11.987445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:19:11.988510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:19:11.988510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:19:11.988510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:19:11.988510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:19:11.988510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:19:11.988510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:19:11.988510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:19:11.988510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:19:11.988510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:19:11.988510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:19:11.988510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:19:11.988510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:19:11.988510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 606/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:19:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:19:11.993651 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfcae2-0860-4067-885f-e8ec30cdca61 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:12.087541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfcae2-0860-4067-885f-e8ec30cdca61 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:12.097036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:12.122800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9d205-bfdd-4e7a-8f2f-f85f90e0a3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:12.123446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9d205-bfdd-4e7a-8f2f-f85f90e0a3ac None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:12.123727 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9d205-bfdd-4e7a-8f2f-f85f90e0a3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:12.124001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9d205-bfdd-4e7a-8f2f-f85f90e0a3ac None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:12.124739 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9d205-bfdd-4e7a-8f2f-f85f90e0a3ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:19:12.125047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9d205-bfdd-4e7a-8f2f-f85f90e0a3ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:12.126425 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9d205-bfdd-4e7a-8f2f-f85f90e0a3ac None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:12.134198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 590/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:19:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:19:12.139956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe93eb-7400-460c-bd26-8bd774c5fed2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:12.238523 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe93eb-7400-460c-bd26-8bd774c5fed2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:12.240035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:12.273997 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1a686-826e-4567-b235-7966f796e314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:12.274863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1a686-826e-4567-b235-7966f796e314 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:12.275148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1a686-826e-4567-b235-7966f796e314 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:12.275424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1a686-826e-4567-b235-7966f796e314 None admin] PATH_INFO: `/v3/users/8a3f8666aad34e24af32e78099ac4b57` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:12.276598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1a686-826e-4567-b235-7966f796e314 None admin] RBAC: Authorizing `identity:get_user(user_id=8a3f8666aad34e24af32e78099ac4b57)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:12.279178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1a686-826e-4567-b235-7966f796e314 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:12.280798 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 607/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Jul 19 15:19:12 2022] GET /identity/v3/users/8a3f8666aad34e24af32e78099ac4b57 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:19:12.286483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9d205-bfdd-4e7a-8f2f-f85f90e0a3ac None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:12.384973 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9d205-bfdd-4e7a-8f2f-f85f90e0a3ac None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:12.418915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4931ee-3cba-4323-ab02-a49c88efa890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:12.419832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4931ee-3cba-4323-ab02-a49c88efa890 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:12.420338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4931ee-3cba-4323-ab02-a49c88efa890 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:12.420827 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4931ee-3cba-4323-ab02-a49c88efa890 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:12.462551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:12.491339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4931ee-3cba-4323-ab02-a49c88efa890 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:12.492714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4931ee-3cba-4323-ab02-a49c88efa890 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:12.590259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 591/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:19:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:19:12.596241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1a686-826e-4567-b235-7966f796e314 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:12.692073 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1a686-826e-4567-b235-7966f796e314 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:12.724926 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28bb97-9025-401d-b6a2-500db7280433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:12.725807 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28bb97-9025-401d-b6a2-500db7280433 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:12.726084 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28bb97-9025-401d-b6a2-500db7280433 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:12.726357 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28bb97-9025-401d-b6a2-500db7280433 None admin] PATH_INFO: `/v3/projects/35c20d8fc8974f2faa4416f6ac927943` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:12.727509 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28bb97-9025-401d-b6a2-500db7280433 None admin] RBAC: Authorizing `identity:get_project(project_id=35c20d8fc8974f2faa4416f6ac927943)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:12.728942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28bb97-9025-401d-b6a2-500db7280433 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:12.731257 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 608/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Jul 19 15:19:12 2022] GET /identity/v3/projects/35c20d8fc8974f2faa4416f6ac927943 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:19:12.737337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4931ee-3cba-4323-ab02-a49c88efa890 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:19:12.804202 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:12.834014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4931ee-3cba-4323-ab02-a49c88efa890 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:19:12.867974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c1af3-b8c7-4875-a063-32d83a4ada3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:19:12.868620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c1af3-b8c7-4875-a063-32d83a4ada3b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:19:12.868902 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c1af3-b8c7-4875-a063-32d83a4ada3b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:19:12.869203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c1af3-b8c7-4875-a063-32d83a4ada3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:19:12.870034 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c1af3-b8c7-4875-a063-32d83a4ada3b None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=35c20d8fc8974f2faa4416f6ac927943, user.id=8a3f8666aad34e24af32e78099ac4b57) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:19:12.870345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c1af3-b8c7-4875-a063-32d83a4ada3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:19:12.871774 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c1af3-b8c7-4875-a063-32d83a4ada3b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:19:12.877492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 592/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:19:12 2022] GET /identity/v3/role_assignments?user.id=8a3f8666aad34e24af32e78099ac4b57&scope.project.id=35c20d8fc8974f2faa4416f6ac927943&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:19:22.720134 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=81 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:19:22.874831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=33 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:07.952481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0a55b-f008-4748-b12e-754d7e2f12c9 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:07.953074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0a55b-f008-4748-b12e-754d7e2f12c9 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:07.953074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0a55b-f008-4748-b12e-754d7e2f12c9 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:07.954762 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 609/1201] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:20:07.960902 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d891a-9476-4330-9f61-3972c2da5a71 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:07.961520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d891a-9476-4330-9f61-3972c2da5a71 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:07.961821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d891a-9476-4330-9f61-3972c2da5a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:07.983090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1a1d891a-9476-4330-9f61-3972c2da5a71 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:20:07.984310 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1a1d891a-9476-4330-9f61-3972c2da5a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:20:08.084209 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 593/1202] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:20:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:20:08.110956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0a55b-f008-4748-b12e-754d7e2f12c9 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:08.252518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0a55b-f008-4748-b12e-754d7e2f12c9 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:08.295855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64ecbe-726e-4948-a160-92e4f67f0256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:08.296793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64ecbe-726e-4948-a160-92e4f67f0256 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:08.297080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64ecbe-726e-4948-a160-92e4f67f0256 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:08.297379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64ecbe-726e-4948-a160-92e4f67f0256 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:08.308951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64ecbe-726e-4948-a160-92e4f67f0256 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:08.310518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64ecbe-726e-4948-a160-92e4f67f0256 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:08.336253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-4e64ecbe-726e-4948-a160-92e4f67f0256 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:08.336253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:08.336253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:08.336253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:08.336253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:08.336253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:08.336253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:08.336253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:08.336253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:08.336253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:08.336253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:08.336253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:08.336253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:20:08.336253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:20:08.336253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:08.337577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:08.338552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:20:08.338552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:20:08.338552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:08.338552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:08.338552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 610/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:08.345192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d891a-9476-4330-9f61-3972c2da5a71 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:08.478706 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d891a-9476-4330-9f61-3972c2da5a71 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:08.519434 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:08.528626 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dea389-b38d-40c7-9c79-380df1317480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:08.529346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dea389-b38d-40c7-9c79-380df1317480 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:08.529629 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dea389-b38d-40c7-9c79-380df1317480 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:08.529901 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dea389-b38d-40c7-9c79-380df1317480 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:08.530638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dea389-b38d-40c7-9c79-380df1317480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:08.530947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dea389-b38d-40c7-9c79-380df1317480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:08.532316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dea389-b38d-40c7-9c79-380df1317480 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:08.545798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 594/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:08.551858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64ecbe-726e-4948-a160-92e4f67f0256 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:08.629014 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:08.700278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64ecbe-726e-4948-a160-92e4f67f0256 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:08.753870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31fbcb-bae2-46fd-8c4c-dbcb4c643026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:08.754766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31fbcb-bae2-46fd-8c4c-dbcb4c643026 None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:08.755146 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31fbcb-bae2-46fd-8c4c-dbcb4c643026 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:08.755656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31fbcb-bae2-46fd-8c4c-dbcb4c643026 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:08.756469 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31fbcb-bae2-46fd-8c4c-dbcb4c643026 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:08.759184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31fbcb-bae2-46fd-8c4c-dbcb4c643026 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:08.793392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 611/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Jul 19 15:20:08 2022] POST /identity/v3/users => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:20:11.134974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5db28-2efe-454b-9c6e-fab604c8a09f None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:11.135550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5db28-2efe-454b-9c6e-fab604c8a09f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:11.135550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5db28-2efe-454b-9c6e-fab604c8a09f None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:11.136905 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 595/1206] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:20:11.142342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52438c-9b54-4ea6-b756-2672fd932679 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:11.142765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52438c-9b54-4ea6-b756-2672fd932679 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:11.142978 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52438c-9b54-4ea6-b756-2672fd932679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:11.161217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-be52438c-9b54-4ea6-b756-2672fd932679 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:20:11.162581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-be52438c-9b54-4ea6-b756-2672fd932679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:20:11.250922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 612/1207] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:20:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:20:11.276071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5db28-2efe-454b-9c6e-fab604c8a09f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:11.404514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5db28-2efe-454b-9c6e-fab604c8a09f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:11.450664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f0a10-69aa-414f-9ee7-dff2e23d5ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:11.451473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f0a10-69aa-414f-9ee7-dff2e23d5ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:11.451761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f0a10-69aa-414f-9ee7-dff2e23d5ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:11.452046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f0a10-69aa-414f-9ee7-dff2e23d5ff8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:11.465966 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f0a10-69aa-414f-9ee7-dff2e23d5ff8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-961f0a10-69aa-414f-9ee7-dff2e23d5ff8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:20:11.471517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:11.472698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:11.474005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:11.474005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:11.474005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:11.474005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:11.474005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:20:11.474005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:20:11.474005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:20:11.474005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:20:11.474005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:20:11.474005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:20:11.474005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:20:11.474005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:11.474005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 596/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:20:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:11.479395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52438c-9b54-4ea6-b756-2672fd932679 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:11.524757 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:11.593652 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52438c-9b54-4ea6-b756-2672fd932679 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:11.633987 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703de2c-9bb6-4690-92cb-d0b55e13fd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:11.634688 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703de2c-9bb6-4690-92cb-d0b55e13fd36 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:11.634982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703de2c-9bb6-4690-92cb-d0b55e13fd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:11.635269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703de2c-9bb6-4690-92cb-d0b55e13fd36 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:11.636028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703de2c-9bb6-4690-92cb-d0b55e13fd36 None admin] RBAC: Adding query filter params (name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:11.636335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703de2c-9bb6-4690-92cb-d0b55e13fd36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:11.637737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703de2c-9bb6-4690-92cb-d0b55e13fd36 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:11.646305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 613/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:20:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:20:11.652801 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f0a10-69aa-414f-9ee7-dff2e23d5ff8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:11.713679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:11.775024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f0a10-69aa-414f-9ee7-dff2e23d5ff8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:11.816371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606bc6b-e870-45e6-9e13-c0569645d122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:11.817400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606bc6b-e870-45e6-9e13-c0569645d122 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:11.817775 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606bc6b-e870-45e6-9e13-c0569645d122 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:11.818098 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606bc6b-e870-45e6-9e13-c0569645d122 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:11.830302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606bc6b-e870-45e6-9e13-c0569645d122 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:11.831755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606bc6b-e870-45e6-9e13-c0569645d122 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:11.849409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-4606bc6b-e870-45e6-9e13-c0569645d122 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:11.849409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:11.849409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:11.849409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:11.849409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:11.849409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:11.849409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:11.849409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:11.849409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:11.849409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:11.849409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:11.849409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:11.849409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:20:11.849409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:20:11.849409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:11.853385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:11.854318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:20:11.854318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:20:11.854318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:11.854318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:11.854318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 597/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:11.856988 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703de2c-9bb6-4690-92cb-d0b55e13fd36 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:11.973038 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:11.979138 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703de2c-9bb6-4690-92cb-d0b55e13fd36 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:12.024741 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086e215-5679-4f7a-bedb-991001d2d21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:12.025709 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086e215-5679-4f7a-bedb-991001d2d21b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:12.026208 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086e215-5679-4f7a-bedb-991001d2d21b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:12.026500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086e215-5679-4f7a-bedb-991001d2d21b None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:12.027335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086e215-5679-4f7a-bedb-991001d2d21b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:12.027644 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086e215-5679-4f7a-bedb-991001d2d21b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:12.029278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086e215-5679-4f7a-bedb-991001d2d21b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:12.043694 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 614/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:20:12.049744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606bc6b-e870-45e6-9e13-c0569645d122 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:12.175085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606bc6b-e870-45e6-9e13-c0569645d122 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:12.177086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:12.219803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f18ec-981d-41ce-aae2-e369d9ab97b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:12.220770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f18ec-981d-41ce-aae2-e369d9ab97b5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:12.221083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f18ec-981d-41ce-aae2-e369d9ab97b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:12.221481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f18ec-981d-41ce-aae2-e369d9ab97b5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:12.231541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f18ec-981d-41ce-aae2-e369d9ab97b5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:12.232977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f18ec-981d-41ce-aae2-e369d9ab97b5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-6a9f18ec-981d-41ce-aae2-e369d9ab97b5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:20:12.250097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:12.251107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:12.252080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:12.252080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:12.252080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:20:12.252080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:20:12.252080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:20:12.252080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:20:12.252080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:20:12.252080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:20:12.252080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 15:20:12.252080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:12.256380 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 598/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:12.257673 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086e215-5679-4f7a-bedb-991001d2d21b None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:12.387040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086e215-5679-4f7a-bedb-991001d2d21b None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:12.431667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a678b0-9fae-4f7a-aa6e-9cb58c85eb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:12.432524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a678b0-9fae-4f7a-aa6e-9cb58c85eb86 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:12.432883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a678b0-9fae-4f7a-aa6e-9cb58c85eb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:12.433307 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a678b0-9fae-4f7a-aa6e-9cb58c85eb86 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:12.434329 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a678b0-9fae-4f7a-aa6e-9cb58c85eb86 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:12.434727 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a678b0-9fae-4f7a-aa6e-9cb58c85eb86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:12.436318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a678b0-9fae-4f7a-aa6e-9cb58c85eb86 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:12.456994 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 615/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Tue Jul 19 15:20:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:12.463422 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f18ec-981d-41ce-aae2-e369d9ab97b5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:12.578072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f18ec-981d-41ce-aae2-e369d9ab97b5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:12.621200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276743e-ce19-46c2-a439-83d5cc92fd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:12.621918 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276743e-ce19-46c2-a439-83d5cc92fd21 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:12.622219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276743e-ce19-46c2-a439-83d5cc92fd21 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:12.622508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276743e-ce19-46c2-a439-83d5cc92fd21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:12.665814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:12.718207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276743e-ce19-46c2-a439-83d5cc92fd21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:12.719714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276743e-ce19-46c2-a439-83d5cc92fd21 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:12.852451 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 599/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:20:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:20:12.857274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a678b0-9fae-4f7a-aa6e-9cb58c85eb86 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:12.880018 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:12.971330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a678b0-9fae-4f7a-aa6e-9cb58c85eb86 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:13.015148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5718d-25e5-49ea-b817-bb5316c47bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:13.015952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5718d-25e5-49ea-b817-bb5316c47bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:13.016233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5718d-25e5-49ea-b817-bb5316c47bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:13.016510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5718d-25e5-49ea-b817-bb5316c47bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:13.032487 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5718d-25e5-49ea-b817-bb5316c47bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:13.034834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5718d-25e5-49ea-b817-bb5316c47bc5 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:13.061252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-26d5718d-25e5-49ea-b817-bb5316c47bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:13.061252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:13.061252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:13.061252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:13.061252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:13.061252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:13.061252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:13.061252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:13.061252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:13.061252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:13.061252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:13.061252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:13.061252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:20:13.061252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:20:13.061252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:13.062407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:13.063476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:20:13.063476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:20:13.063476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:13.063476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:13.063476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 616/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:13.068853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276743e-ce19-46c2-a439-83d5cc92fd21 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:13.136783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:13.183507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276743e-ce19-46c2-a439-83d5cc92fd21 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:13.225319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a5169-cc62-416c-bc72-235a76b8f759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:13.226475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a5169-cc62-416c-bc72-235a76b8f759 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:13.226957 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a5169-cc62-416c-bc72-235a76b8f759 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:13.227410 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a5169-cc62-416c-bc72-235a76b8f759 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:13.228314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a5169-cc62-416c-bc72-235a76b8f759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:13.228798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a5169-cc62-416c-bc72-235a76b8f759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:13.230375 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a5169-cc62-416c-bc72-235a76b8f759 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:13.243665 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 600/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:20:13.250146 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5718d-25e5-49ea-b817-bb5316c47bc5 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:13.338819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:13.385923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5718d-25e5-49ea-b817-bb5316c47bc5 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:13.431038 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9a500-9f58-406e-b6a0-98363eeef797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:13.431938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9a500-9f58-406e-b6a0-98363eeef797 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:13.432316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9a500-9f58-406e-b6a0-98363eeef797 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:13.432693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9a500-9f58-406e-b6a0-98363eeef797 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:13.440053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9a500-9f58-406e-b6a0-98363eeef797 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:13.441702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9a500-9f58-406e-b6a0-98363eeef797 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:13.456502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-09a9a500-9f58-406e-b6a0-98363eeef797 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:20:13.456502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:13.456502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:13.456502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:13.456502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:13.456502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:13.456502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:13.456502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:13.456502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:13.456502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:13.456502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:13.456502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:13.456502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:20:13.456502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:20:13.456502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:13.457935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:13.458851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:20:13.458851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:20:13.458851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:20:13.458851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:20:13.458851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:20:13.458851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:20:13.458851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:20:13.458851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:13.458851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 617/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:20:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:13.464344 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a5169-cc62-416c-bc72-235a76b8f759 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:13.587717 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a5169-cc62-416c-bc72-235a76b8f759 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:13.620899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:13.635620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5626df30-ab7e-4f55-946c-741be714dc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:13.636647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626df30-ab7e-4f55-946c-741be714dc7a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:13.636939 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626df30-ab7e-4f55-946c-741be714dc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:13.637281 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626df30-ab7e-4f55-946c-741be714dc7a None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:13.638445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626df30-ab7e-4f55-946c-741be714dc7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:13.638883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626df30-ab7e-4f55-946c-741be714dc7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:13.640612 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626df30-ab7e-4f55-946c-741be714dc7a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:13.661175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 601/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:20:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:13.662523 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9a500-9f58-406e-b6a0-98363eeef797 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:13.786880 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9a500-9f58-406e-b6a0-98363eeef797 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:13.825695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:13.834989 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da1499-f64f-4a73-94b8-c0e5f9e5a9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:13.836457 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da1499-f64f-4a73-94b8-c0e5f9e5a9cb None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:13.836744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da1499-f64f-4a73-94b8-c0e5f9e5a9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:13.837079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da1499-f64f-4a73-94b8-c0e5f9e5a9cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:13.838065 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da1499-f64f-4a73-94b8-c0e5f9e5a9cb None admin] RBAC: Adding query filter params (role.id=c8578ede4e7041ecaeba79c255cc650b, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=6002f32936944b0db6fd04f3851836db) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:13.838379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da1499-f64f-4a73-94b8-c0e5f9e5a9cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:13.840164 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da1499-f64f-4a73-94b8-c0e5f9e5a9cb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:13.847636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 618/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:20:13 2022] GET /identity/v3/role_assignments?user.id=6002f32936944b0db6fd04f3851836db&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=c8578ede4e7041ecaeba79c255cc650b => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:16.091561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f14280-af60-4db5-a03b-6fb8ca90ad38 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:16.092118 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f14280-af60-4db5-a03b-6fb8ca90ad38 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:16.092118 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f14280-af60-4db5-a03b-6fb8ca90ad38 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:16.093549 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 602/1220] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:20:16.098775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e7e5d-cfc5-42a8-be38-dba450682037 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:16.099176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e7e5d-cfc5-42a8-be38-dba450682037 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:16.099363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e7e5d-cfc5-42a8-be38-dba450682037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:16.117060 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-669e7e5d-cfc5-42a8-be38-dba450682037 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:20:16.118316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-669e7e5d-cfc5-42a8-be38-dba450682037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:20:16.189211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 619/1221] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:20:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:20:16.220722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f14280-af60-4db5-a03b-6fb8ca90ad38 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:16.347863 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f14280-af60-4db5-a03b-6fb8ca90ad38 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:16.382435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c89ac9-528e-406b-a1fe-0c8c41174365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:16.383234 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c89ac9-528e-406b-a1fe-0c8c41174365 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:16.383518 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c89ac9-528e-406b-a1fe-0c8c41174365 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:16.383790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c89ac9-528e-406b-a1fe-0c8c41174365 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:16.394049 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c89ac9-528e-406b-a1fe-0c8c41174365 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e2c89ac9-528e-406b-a1fe-0c8c41174365 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:20:16.403545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:16.404713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:16.405740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:16.405740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:16.405740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:16.405740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:16.405740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:20:16.405740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:20:16.405740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:20:16.405740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:20:16.405740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:20:16.405740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:20:16.405740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:20:16.405740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:16.405740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 603/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:20:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:20:16.412464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e7e5d-cfc5-42a8-be38-dba450682037 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:16.523597 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e7e5d-cfc5-42a8-be38-dba450682037 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:16.547034 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:16.558875 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e29571-4ec4-4000-83e9-4411333f2260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:16.559545 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e29571-4ec4-4000-83e9-4411333f2260 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:16.559826 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e29571-4ec4-4000-83e9-4411333f2260 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:16.560098 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e29571-4ec4-4000-83e9-4411333f2260 None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:16.560843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e29571-4ec4-4000-83e9-4411333f2260 None admin] RBAC: Adding query filter params (name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:16.561157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e29571-4ec4-4000-83e9-4411333f2260 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:16.562555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e29571-4ec4-4000-83e9-4411333f2260 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:16.571084 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 620/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:20:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:16.577514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c89ac9-528e-406b-a1fe-0c8c41174365 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:16.624667 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:16.689987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c89ac9-528e-406b-a1fe-0c8c41174365 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:16.731460 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39a1bd-604d-43f8-99d1-c61a6fddcfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:16.732398 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39a1bd-604d-43f8-99d1-c61a6fddcfe5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:16.732715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39a1bd-604d-43f8-99d1-c61a6fddcfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:16.733061 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39a1bd-604d-43f8-99d1-c61a6fddcfe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:16.748233 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39a1bd-604d-43f8-99d1-c61a6fddcfe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:16.749966 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39a1bd-604d-43f8-99d1-c61a6fddcfe5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:16.771613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3e39a1bd-604d-43f8-99d1-c61a6fddcfe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:16.771613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:16.771613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:16.771613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:16.771613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:16.771613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:16.771613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:16.771613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:16.771613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:16.771613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:16.771613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:16.771613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:16.771613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:20:16.771613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:20:16.771613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:16.772829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:16.774076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:20:16.774076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:20:16.774076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:16.774076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:16.774076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 604/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:16.779849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e29571-4ec4-4000-83e9-4411333f2260 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:16.891336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e29571-4ec4-4000-83e9-4411333f2260 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:16.930783 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df107d-946c-44a3-b3cc-f04ccff8e80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:16.931659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df107d-946c-44a3-b3cc-f04ccff8e80f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:16.931943 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df107d-946c-44a3-b3cc-f04ccff8e80f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:16.932220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df107d-946c-44a3-b3cc-f04ccff8e80f None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:16.933116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df107d-946c-44a3-b3cc-f04ccff8e80f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:16.933455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df107d-946c-44a3-b3cc-f04ccff8e80f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:16.935110 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df107d-946c-44a3-b3cc-f04ccff8e80f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:16.950052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 621/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:20:16.956644 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39a1bd-604d-43f8-99d1-c61a6fddcfe5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:17.069067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:17.077937 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39a1bd-604d-43f8-99d1-c61a6fddcfe5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:17.115249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69eb4b-3117-4abd-be31-920801ec98cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:17.116104 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69eb4b-3117-4abd-be31-920801ec98cb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:17.116501 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69eb4b-3117-4abd-be31-920801ec98cb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:17.116783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69eb4b-3117-4abd-be31-920801ec98cb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:17.125173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69eb4b-3117-4abd-be31-920801ec98cb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:17.126814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69eb4b-3117-4abd-be31-920801ec98cb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3e69eb4b-3117-4abd-be31-920801ec98cb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:20:17.142770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:17.143969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:17.145719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:17.145719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:17.145719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:20:17.145719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:20:17.145719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:20:17.145719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:20:17.145719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:20:17.145719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:20:17.145719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 15:20:17.145719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:17.145719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 605/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:17.150838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df107d-946c-44a3-b3cc-f04ccff8e80f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:17.204004 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:17.271217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df107d-946c-44a3-b3cc-f04ccff8e80f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:17.311536 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90429ac1-0d7c-42d8-a4ea-04de1b059ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:17.312259 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90429ac1-0d7c-42d8-a4ea-04de1b059ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:17.312548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90429ac1-0d7c-42d8-a4ea-04de1b059ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:17.312831 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90429ac1-0d7c-42d8-a4ea-04de1b059ad1 None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:17.313842 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90429ac1-0d7c-42d8-a4ea-04de1b059ad1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:17.314185 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90429ac1-0d7c-42d8-a4ea-04de1b059ad1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:17.315663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90429ac1-0d7c-42d8-a4ea-04de1b059ad1 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:17.335859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 622/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Tue Jul 19 15:20:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:17.341870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69eb4b-3117-4abd-be31-920801ec98cb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:17.456122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69eb4b-3117-4abd-be31-920801ec98cb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:17.491744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd62f2-5f38-477b-9030-f33ed56c6c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:17.492576 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd62f2-5f38-477b-9030-f33ed56c6c64 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:17.492964 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd62f2-5f38-477b-9030-f33ed56c6c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:17.493277 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd62f2-5f38-477b-9030-f33ed56c6c64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:17.505952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd62f2-5f38-477b-9030-f33ed56c6c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:17.508638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd62f2-5f38-477b-9030-f33ed56c6c64 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:17.528995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-fccd62f2-5f38-477b-9030-f33ed56c6c64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:17.528995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:17.528995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:17.528995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:17.528995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:17.528995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:17.528995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:17.528995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:17.528995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:17.528995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:17.528995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:17.528995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:17.528995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:20:17.528995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:20:17.528995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:17.529981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:17.531254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:20:17.531254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:20:17.531254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:17.531254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:17.531254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 606/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:20:17.536116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90429ac1-0d7c-42d8-a4ea-04de1b059ad1 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:17.643694 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90429ac1-0d7c-42d8-a4ea-04de1b059ad1 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:17.646375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:17.681561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9f098-a508-4a9f-a35f-ef43d73b9ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:17.682251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9f098-a508-4a9f-a35f-ef43d73b9ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:17.682532 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9f098-a508-4a9f-a35f-ef43d73b9ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:17.682805 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9f098-a508-4a9f-a35f-ef43d73b9ca2 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:17.683534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9f098-a508-4a9f-a35f-ef43d73b9ca2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:17.683840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9f098-a508-4a9f-a35f-ef43d73b9ca2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:17.685224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9f098-a508-4a9f-a35f-ef43d73b9ca2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:17.699234 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 623/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:17.705833 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd62f2-5f38-477b-9030-f33ed56c6c64 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:17.718714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:17.824766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd62f2-5f38-477b-9030-f33ed56c6c64 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:17.863263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d9161-2d91-4b84-a5c8-4e19ef2ca502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:17.864074 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d9161-2d91-4b84-a5c8-4e19ef2ca502 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:17.864377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d9161-2d91-4b84-a5c8-4e19ef2ca502 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:17.864801 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d9161-2d91-4b84-a5c8-4e19ef2ca502 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:17.873534 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d9161-2d91-4b84-a5c8-4e19ef2ca502 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:17.875003 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d9161-2d91-4b84-a5c8-4e19ef2ca502 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:17.891109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-4d4d9161-2d91-4b84-a5c8-4e19ef2ca502 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:20:17.891109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:17.891109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:17.891109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:17.891109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:17.891109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:17.891109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:17.891109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:17.891109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:17.891109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:17.891109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:17.891109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:17.891109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:20:17.891109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:20:17.891109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:17.892283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:17.893466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:20:17.893466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:20:17.893466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:20:17.893466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:20:17.893466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:20:17.893466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:20:17.893466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:20:17.893466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:17.893466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 607/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:20:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:20:17.899372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9f098-a508-4a9f-a35f-ef43d73b9ca2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:18.019127 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9f098-a508-4a9f-a35f-ef43d73b9ca2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:18.058538 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376adad7-7853-4b66-b790-fc1a471287fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:18.059320 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376adad7-7853-4b66-b790-fc1a471287fd None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:18.059576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376adad7-7853-4b66-b790-fc1a471287fd None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:18.060116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376adad7-7853-4b66-b790-fc1a471287fd None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:18.061121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376adad7-7853-4b66-b790-fc1a471287fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:18.061650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376adad7-7853-4b66-b790-fc1a471287fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:18.063102 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376adad7-7853-4b66-b790-fc1a471287fd None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:18.076894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 624/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:20:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:18.083275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d9161-2d91-4b84-a5c8-4e19ef2ca502 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:18.156673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:18.198337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d9161-2d91-4b84-a5c8-4e19ef2ca502 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:18.239965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b705862-b1db-41b9-a1d8-f6f40bc288f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:18.240907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b705862-b1db-41b9-a1d8-f6f40bc288f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:18.241719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b705862-b1db-41b9-a1d8-f6f40bc288f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:18.242509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b705862-b1db-41b9-a1d8-f6f40bc288f6 None admin] PATH_INFO: `/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/6002f32936944b0db6fd04f3851836db/roles/c8578ede4e7041ecaeba79c255cc650b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:18.259540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b705862-b1db-41b9-a1d8-f6f40bc288f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5f51e47b8e34bf5ac822d8c93853e30, user_id=6002f32936944b0db6fd04f3851836db, role_id=c8578ede4e7041ecaeba79c255cc650b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:18.261028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b705862-b1db-41b9-a1d8-f6f40bc288f6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:18.273933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 608/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:20:18 2022] PUT /identity/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/6002f32936944b0db6fd04f3851836db/roles/c8578ede4e7041ecaeba79c255cc650b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:20:20.467432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17150770-d012-442c-8317-979e81699819 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:20.468018 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17150770-d012-442c-8317-979e81699819 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:20.468018 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17150770-d012-442c-8317-979e81699819 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:20.469384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 625/1233] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:20:20.474655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414338a7-cd07-4d97-96ba-3643946472a8 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:20.475038 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414338a7-cd07-4d97-96ba-3643946472a8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:20.475219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414338a7-cd07-4d97-96ba-3643946472a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:20.493638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-414338a7-cd07-4d97-96ba-3643946472a8 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:20:20.494823 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-414338a7-cd07-4d97-96ba-3643946472a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:20:20.564922 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 609/1234] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:20:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:20:20.588534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17150770-d012-442c-8317-979e81699819 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:20.614800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:20.704499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17150770-d012-442c-8317-979e81699819 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:20.742194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f163f-dab5-4b4b-b362-ed60f13b7a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:20.744001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f163f-dab5-4b4b-b362-ed60f13b7a0e None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:20.744345 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f163f-dab5-4b4b-b362-ed60f13b7a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:20.744643 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f163f-dab5-4b4b-b362-ed60f13b7a0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:20.757419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26f163f-dab5-4b4b-b362-ed60f13b7a0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-f26f163f-dab5-4b4b-b362-ed60f13b7a0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:20:20.765189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:20.766348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:20.773163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:20.773163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:20.773163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:20.773163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:20.773163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:20:20.773163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:20:20.773163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:20:20.773163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:20:20.773163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:20:20.773163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:20:20.773163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 15:20:20.773163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:20.773163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 626/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Jul 19 15:20:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:20.775231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414338a7-cd07-4d97-96ba-3643946472a8 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:20.823061 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:20.884189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414338a7-cd07-4d97-96ba-3643946472a8 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:20.926624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d90be-9302-4439-8f75-9b83c4b238a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:20.927326 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d90be-9302-4439-8f75-9b83c4b238a8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:20.927604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d90be-9302-4439-8f75-9b83c4b238a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:20.927886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d90be-9302-4439-8f75-9b83c4b238a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:20.928653 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d90be-9302-4439-8f75-9b83c4b238a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:20.928956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d90be-9302-4439-8f75-9b83c4b238a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:20.931208 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d90be-9302-4439-8f75-9b83c4b238a8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:20.940436 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 610/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Jul 19 15:20:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:20.946777 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f163f-dab5-4b4b-b362-ed60f13b7a0e None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:21.049779 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:21.066741 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f163f-dab5-4b4b-b362-ed60f13b7a0e None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:21.105040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc17de-862e-47bf-95a4-984e2c380e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:21.105781 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc17de-862e-47bf-95a4-984e2c380e39 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:21.106064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc17de-862e-47bf-95a4-984e2c380e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:21.106340 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc17de-862e-47bf-95a4-984e2c380e39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:21.118733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc17de-862e-47bf-95a4-984e2c380e39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:21.120325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc17de-862e-47bf-95a4-984e2c380e39 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:21.139347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-b4fc17de-862e-47bf-95a4-984e2c380e39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:21.139347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:21.139347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:21.139347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:21.139347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:21.139347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:21.139347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:21.139347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:21.139347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:21.139347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:21.139347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:21.139347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:21.139347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:20:21.139347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:20:21.139347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:21.140325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:21.141283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:20:21.141283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:20:21.141283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:21.141283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:21.141283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 627/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:21.146878 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d90be-9302-4439-8f75-9b83c4b238a8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:21.259654 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d90be-9302-4439-8f75-9b83c4b238a8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:21.269671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:21.297941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53953c3a-438a-45ca-b0d4-da8fcbdcf5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:21.298612 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53953c3a-438a-45ca-b0d4-da8fcbdcf5bf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:21.298908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53953c3a-438a-45ca-b0d4-da8fcbdcf5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:21.299189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53953c3a-438a-45ca-b0d4-da8fcbdcf5bf None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:21.299918 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53953c3a-438a-45ca-b0d4-da8fcbdcf5bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:21.300229 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53953c3a-438a-45ca-b0d4-da8fcbdcf5bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:21.301637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53953c3a-438a-45ca-b0d4-da8fcbdcf5bf None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:21.314948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 611/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:21.320983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc17de-862e-47bf-95a4-984e2c380e39 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:21.425917 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc17de-862e-47bf-95a4-984e2c380e39 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:21.464414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040c710-5bcb-4b5c-96e9-9f5b5f741a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:21.465336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040c710-5bcb-4b5c-96e9-9f5b5f741a96 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:21.465632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040c710-5bcb-4b5c-96e9-9f5b5f741a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:21.465914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040c710-5bcb-4b5c-96e9-9f5b5f741a96 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:21.469065 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:21.476413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040c710-5bcb-4b5c-96e9-9f5b5f741a96 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:21.478406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040c710-5bcb-4b5c-96e9-9f5b5f741a96 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-b040c710-5bcb-4b5c-96e9-9f5b5f741a96 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:20:21.494541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:21.495706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:21.496737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:21.496737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:21.496737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:20:21.496737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:20:21.496737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:20:21.496737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:20:21.496737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:20:21.496737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:20:21.496737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 15:20:21.496737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:21.496737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 628/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:21.503072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53953c3a-438a-45ca-b0d4-da8fcbdcf5bf None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:21.614402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53953c3a-438a-45ca-b0d4-da8fcbdcf5bf None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:21.650776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b214f-e4b1-4d3f-bc31-a475cdb36966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:21.651454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b214f-e4b1-4d3f-bc31-a475cdb36966 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:21.651740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b214f-e4b1-4d3f-bc31-a475cdb36966 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:21.652017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b214f-e4b1-4d3f-bc31-a475cdb36966 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:21.652865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b214f-e4b1-4d3f-bc31-a475cdb36966 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:21.653206 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b214f-e4b1-4d3f-bc31-a475cdb36966 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:21.654655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b214f-e4b1-4d3f-bc31-a475cdb36966 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:21.676550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 612/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Tue Jul 19 15:20:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:21.682620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040c710-5bcb-4b5c-96e9-9f5b5f741a96 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:21.806794 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040c710-5bcb-4b5c-96e9-9f5b5f741a96 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:21.846473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e08221-4e11-4627-8d37-27bf979b370b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:21.847132 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e08221-4e11-4627-8d37-27bf979b370b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:21.847406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e08221-4e11-4627-8d37-27bf979b370b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:21.847685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e08221-4e11-4627-8d37-27bf979b370b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:21.920611 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:21.927858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e08221-4e11-4627-8d37-27bf979b370b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:21.930057 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e08221-4e11-4627-8d37-27bf979b370b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:22.045946 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 629/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:20:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:20:22.052131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b214f-e4b1-4d3f-bc31-a475cdb36966 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:22.082052 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:22.162920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b214f-e4b1-4d3f-bc31-a475cdb36966 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:22.203785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68f0be-901b-44f8-aa8d-6edfac3d921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:22.204640 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68f0be-901b-44f8-aa8d-6edfac3d921e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:22.204938 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68f0be-901b-44f8-aa8d-6edfac3d921e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:22.205254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68f0be-901b-44f8-aa8d-6edfac3d921e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:22.219662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68f0be-901b-44f8-aa8d-6edfac3d921e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:22.221458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68f0be-901b-44f8-aa8d-6edfac3d921e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:22.242551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-be68f0be-901b-44f8-aa8d-6edfac3d921e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:22.242551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:22.242551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:22.242551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:22.242551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:22.242551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:22.242551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:22.242551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:22.242551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:22.242551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:22.242551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:22.242551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:22.242551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:20:22.242551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:20:22.242551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:22.243487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:22.244740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:20:22.244740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:20:22.244740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:22.244740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:22.244740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 613/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:22.249706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e08221-4e11-4627-8d37-27bf979b370b None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:22.355775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:22.362986 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e08221-4e11-4627-8d37-27bf979b370b None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:22.401774 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1a00b-c23a-4edc-bf9f-a4e52055d988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:22.402507 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1a00b-c23a-4edc-bf9f-a4e52055d988 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:22.402713 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1a00b-c23a-4edc-bf9f-a4e52055d988 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:22.402979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1a00b-c23a-4edc-bf9f-a4e52055d988 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:22.403717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1a00b-c23a-4edc-bf9f-a4e52055d988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:22.404018 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1a00b-c23a-4edc-bf9f-a4e52055d988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:22.405407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1a00b-c23a-4edc-bf9f-a4e52055d988 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:22.418410 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 630/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:20:22.425396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68f0be-901b-44f8-aa8d-6edfac3d921e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:22.523533 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:22.550296 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68f0be-901b-44f8-aa8d-6edfac3d921e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:22.587529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580af610-884e-4b7f-bf60-64f9a5c3795f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:22.588330 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580af610-884e-4b7f-bf60-64f9a5c3795f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:22.588609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580af610-884e-4b7f-bf60-64f9a5c3795f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:22.588881 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580af610-884e-4b7f-bf60-64f9a5c3795f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:22.595915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580af610-884e-4b7f-bf60-64f9a5c3795f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:22.597433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580af610-884e-4b7f-bf60-64f9a5c3795f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:22.611658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-580af610-884e-4b7f-bf60-64f9a5c3795f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:20:22.611658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:22.611658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:22.611658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:22.611658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:22.611658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:22.611658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:22.611658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:22.611658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:22.611658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:22.611658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:22.611658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:22.611658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:20:22.611658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:20:22.611658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:22.612623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:22.613591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:20:22.613591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:20:22.613591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:20:22.613591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:20:22.613591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:20:22.613591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:20:22.613591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:20:22.613591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:22.613591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 614/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:20:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:22.619520 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1a00b-c23a-4edc-bf9f-a4e52055d988 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:22.737693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1a00b-c23a-4edc-bf9f-a4e52055d988 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:22.776577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d508831-f8ec-46d9-8370-b862c2541028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:22.777221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d508831-f8ec-46d9-8370-b862c2541028 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:22.777567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d508831-f8ec-46d9-8370-b862c2541028 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:22.777871 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d508831-f8ec-46d9-8370-b862c2541028 None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:22.778797 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d508831-f8ec-46d9-8370-b862c2541028 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:22.779141 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d508831-f8ec-46d9-8370-b862c2541028 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:22.780593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d508831-f8ec-46d9-8370-b862c2541028 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:22.795304 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 631/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:20:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:22.801427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580af610-884e-4b7f-bf60-64f9a5c3795f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:22.919028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580af610-884e-4b7f-bf60-64f9a5c3795f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:22.960902 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47adf011-f2cc-4e06-8971-92156d6b466c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:22.961709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adf011-f2cc-4e06-8971-92156d6b466c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:22.961994 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adf011-f2cc-4e06-8971-92156d6b466c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:22.963048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adf011-f2cc-4e06-8971-92156d6b466c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:22.963480 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47adf011-f2cc-4e06-8971-92156d6b466c None admin] RBAC: Adding query filter params (role.id=c8578ede4e7041ecaeba79c255cc650b, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=6002f32936944b0db6fd04f3851836db) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:22.963870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47adf011-f2cc-4e06-8971-92156d6b466c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:22.966032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47adf011-f2cc-4e06-8971-92156d6b466c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:22.974185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 615/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:20:22 2022] GET /identity/v3/role_assignments?user.id=6002f32936944b0db6fd04f3851836db&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=c8578ede4e7041ecaeba79c255cc650b => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:20:25.088313 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddd37b-d445-4b02-8956-ca6bda36d479 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:25.088889 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddd37b-d445-4b02-8956-ca6bda36d479 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:25.088889 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddd37b-d445-4b02-8956-ca6bda36d479 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:25.090263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 632/1247] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:20:25.095935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813e3dc-6868-4fdb-ad4d-38abbf3939de None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:25.096407 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813e3dc-6868-4fdb-ad4d-38abbf3939de None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:25.096634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813e3dc-6868-4fdb-ad4d-38abbf3939de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:25.103645 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:25.117789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b813e3dc-6868-4fdb-ad4d-38abbf3939de None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:20:25.119125 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b813e3dc-6868-4fdb-ad4d-38abbf3939de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:20:25.195137 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 616/1248] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:20:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 15:20:25.218865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddd37b-d445-4b02-8956-ca6bda36d479 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:25.242649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:25.345967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddd37b-d445-4b02-8956-ca6bda36d479 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:25.390577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb00fa-bd24-461b-b99d-5bbaf0ab3e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:25.391581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb00fa-bd24-461b-b99d-5bbaf0ab3e26 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:25.391875 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb00fa-bd24-461b-b99d-5bbaf0ab3e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:25.392157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb00fa-bd24-461b-b99d-5bbaf0ab3e26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:25.404121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb00fa-bd24-461b-b99d-5bbaf0ab3e26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-fbeb00fa-bd24-461b-b99d-5bbaf0ab3e26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:20:25.411913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:25.414197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:25.415056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:25.415056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:25.415056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:25.415056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:25.415056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:20:25.415056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:20:25.415056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:20:25.415056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:20:25.415056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:20:25.415056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:20:25.415056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:20:25.415056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:25.415056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 633/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:20:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 15:20:25.420367 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813e3dc-6868-4fdb-ad4d-38abbf3939de None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:25.536802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813e3dc-6868-4fdb-ad4d-38abbf3939de None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:25.574589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:25.579981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e960f-d366-44f8-b265-385b0089b8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:25.580660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e960f-d366-44f8-b265-385b0089b8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:25.580944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e960f-d366-44f8-b265-385b0089b8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:25.581263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e960f-d366-44f8-b265-385b0089b8b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:25.582056 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e960f-d366-44f8-b265-385b0089b8b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:25.582327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e960f-d366-44f8-b265-385b0089b8b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:25.583695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e960f-d366-44f8-b265-385b0089b8b2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:25.594676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 617/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:20:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:25.601317 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb00fa-bd24-461b-b99d-5bbaf0ab3e26 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:25.690333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:25.714927 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb00fa-bd24-461b-b99d-5bbaf0ab3e26 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:25.754943 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be201873-6cd1-4e60-96ac-76eeade950ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:25.755752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201873-6cd1-4e60-96ac-76eeade950ea None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:25.756035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201873-6cd1-4e60-96ac-76eeade950ea None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:25.756383 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201873-6cd1-4e60-96ac-76eeade950ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:25.767435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be201873-6cd1-4e60-96ac-76eeade950ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:25.768879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be201873-6cd1-4e60-96ac-76eeade950ea None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:25.787395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-be201873-6cd1-4e60-96ac-76eeade950ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:25.787395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:25.787395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:25.787395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:25.787395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:25.787395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:25.787395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:25.787395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:25.787395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:25.787395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:25.787395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:25.787395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:25.787395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:20:25.787395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:20:25.787395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:25.788648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:25.789621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:20:25.789621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:20:25.789621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:25.789621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:25.789621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 634/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:25.794776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e960f-d366-44f8-b265-385b0089b8b2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:25.917223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e960f-d366-44f8-b265-385b0089b8b2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:25.958761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f041f08-566f-454c-abe6-300a19070413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:25.960071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f041f08-566f-454c-abe6-300a19070413 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:25.960380 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f041f08-566f-454c-abe6-300a19070413 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:25.960646 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f041f08-566f-454c-abe6-300a19070413 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:25.961497 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f041f08-566f-454c-abe6-300a19070413 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:25.961839 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f041f08-566f-454c-abe6-300a19070413 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:25.963452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f041f08-566f-454c-abe6-300a19070413 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:25.979679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 618/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:25.986739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be201873-6cd1-4e60-96ac-76eeade950ea None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:26.104714 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be201873-6cd1-4e60-96ac-76eeade950ea None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:26.135258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:26.149713 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cb600-3389-4033-ad09-645263ba35fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:26.151094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cb600-3389-4033-ad09-645263ba35fb None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:26.151586 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cb600-3389-4033-ad09-645263ba35fb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:26.152055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cb600-3389-4033-ad09-645263ba35fb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:26.165019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6cb600-3389-4033-ad09-645263ba35fb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:26.167774 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6cb600-3389-4033-ad09-645263ba35fb None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-4c6cb600-3389-4033-ad09-645263ba35fb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:20:26.190363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:26.191416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:26.192366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:26.192366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:26.192366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:20:26.192366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:20:26.192366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:20:26.192366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:20:26.192366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:20:26.192366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:20:26.192366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 15:20:26.192366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:26.192366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 635/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:26.198580 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f041f08-566f-454c-abe6-300a19070413 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:26.256067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:26.305274 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f041f08-566f-454c-abe6-300a19070413 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:26.345842 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345a863-e154-4528-afc5-f7e681c92643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:26.346649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345a863-e154-4528-afc5-f7e681c92643 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:26.346947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345a863-e154-4528-afc5-f7e681c92643 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:26.347291 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345a863-e154-4528-afc5-f7e681c92643 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:26.348403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345a863-e154-4528-afc5-f7e681c92643 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:26.348739 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345a863-e154-4528-afc5-f7e681c92643 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:26.350617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345a863-e154-4528-afc5-f7e681c92643 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:26.369847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 619/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Tue Jul 19 15:20:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:26.376359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cb600-3389-4033-ad09-645263ba35fb None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:26.485417 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cb600-3389-4033-ad09-645263ba35fb None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:26.520980 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a290f0-a773-4829-ba82-7307fd2ac2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:26.521660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a290f0-a773-4829-ba82-7307fd2ac2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:26.522527 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a290f0-a773-4829-ba82-7307fd2ac2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:26.522810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a290f0-a773-4829-ba82-7307fd2ac2f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:26.582295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:26.595818 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a290f0-a773-4829-ba82-7307fd2ac2f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:26.597352 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a290f0-a773-4829-ba82-7307fd2ac2f5 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:26.703108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 636/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:20:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 15:20:26.709021 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345a863-e154-4528-afc5-f7e681c92643 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:26.817387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345a863-e154-4528-afc5-f7e681c92643 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:26.833026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:26.857318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5767b-341b-4308-b691-5d299590dd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:26.858113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5767b-341b-4308-b691-5d299590dd50 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:26.858415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5767b-341b-4308-b691-5d299590dd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:26.858674 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5767b-341b-4308-b691-5d299590dd50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:26.869590 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5767b-341b-4308-b691-5d299590dd50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:26.871440 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5767b-341b-4308-b691-5d299590dd50 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:26.889858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-21e5767b-341b-4308-b691-5d299590dd50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:26.889858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:26.889858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:26.889858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:26.889858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:26.889858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:26.889858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:26.889858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:26.889858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:26.889858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:26.889858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:26.889858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:26.889858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:20:26.889858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:20:26.889858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:26.891006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:26.892306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:20:26.892306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:20:26.892306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:26.892306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:26.892306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 620/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:26.897487 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a290f0-a773-4829-ba82-7307fd2ac2f5 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:26.992044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:27.007299 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a290f0-a773-4829-ba82-7307fd2ac2f5 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:27.047046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f64e1-370e-4d1b-92d3-531bd91e6304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:27.047734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f64e1-370e-4d1b-92d3-531bd91e6304 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:27.048015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f64e1-370e-4d1b-92d3-531bd91e6304 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:27.048290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f64e1-370e-4d1b-92d3-531bd91e6304 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:27.049043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f64e1-370e-4d1b-92d3-531bd91e6304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:27.049394 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f64e1-370e-4d1b-92d3-531bd91e6304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:27.050835 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f64e1-370e-4d1b-92d3-531bd91e6304 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:27.065889 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 637/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:27.071740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5767b-341b-4308-b691-5d299590dd50 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:27.179413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5767b-341b-4308-b691-5d299590dd50 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:27.217434 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46548e2e-2475-46da-a877-c33d8b3c56be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:27.218242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46548e2e-2475-46da-a877-c33d8b3c56be None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:27.218525 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46548e2e-2475-46da-a877-c33d8b3c56be None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:27.218800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46548e2e-2475-46da-a877-c33d8b3c56be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:27.227472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46548e2e-2475-46da-a877-c33d8b3c56be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:27.228933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46548e2e-2475-46da-a877-c33d8b3c56be None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:27.243993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-46548e2e-2475-46da-a877-c33d8b3c56be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:20:27.243993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:27.243993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:27.243993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:27.243993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:27.243993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:27.243993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:27.243993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:27.243993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:27.243993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:27.243993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:27.243993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:27.243993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:20:27.243993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:20:27.243993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:27.245057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:27.246329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:20:27.246329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:20:27.246329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:20:27.246329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:20:27.246329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:20:27.246329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:20:27.246329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:20:27.246329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:27.246329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 621/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:20:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:27.251172 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f64e1-370e-4d1b-92d3-531bd91e6304 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:27.356443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f64e1-370e-4d1b-92d3-531bd91e6304 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:27.391453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cb4d1-c692-47ba-80c5-8e6294262f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:27.392250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cb4d1-c692-47ba-80c5-8e6294262f8b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:27.392601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cb4d1-c692-47ba-80c5-8e6294262f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:27.392943 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cb4d1-c692-47ba-80c5-8e6294262f8b None admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:27.393869 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cb4d1-c692-47ba-80c5-8e6294262f8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:27.394501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cb4d1-c692-47ba-80c5-8e6294262f8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:27.396030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cb4d1-c692-47ba-80c5-8e6294262f8b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:27.410056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 638/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:20:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:27.416992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46548e2e-2475-46da-a877-c33d8b3c56be None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:27.445104 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:27.521097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46548e2e-2475-46da-a877-c33d8b3c56be None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:27.556236 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93a738-64ab-4690-a082-3b7fc6ac6ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:27.556898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93a738-64ab-4690-a082-3b7fc6ac6ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:27.557211 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93a738-64ab-4690-a082-3b7fc6ac6ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:27.557492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93a738-64ab-4690-a082-3b7fc6ac6ce5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:27.558326 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93a738-64ab-4690-a082-3b7fc6ac6ce5 None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=6002f32936944b0db6fd04f3851836db) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:27.558636 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93a738-64ab-4690-a082-3b7fc6ac6ce5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:27.560057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93a738-64ab-4690-a082-3b7fc6ac6ce5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:27.566039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 622/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:20:27 2022] GET /identity/v3/role_assignments?user.id=6002f32936944b0db6fd04f3851836db&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:29.576618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cd2f7-9c1b-40d3-ad0d-bebaf5c0246c None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:29.577132 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cd2f7-9c1b-40d3-ad0d-bebaf5c0246c None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:29.577132 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cd2f7-9c1b-40d3-ad0d-bebaf5c0246c None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:29.578489 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 639/1261] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:20:29.583460 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15b221-d88f-4944-8a5f-ed7544610d94 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:29.583879 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15b221-d88f-4944-8a5f-ed7544610d94 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:29.584609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15b221-d88f-4944-8a5f-ed7544610d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:29.601541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8e15b221-d88f-4944-8a5f-ed7544610d94 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:20:29.602711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8e15b221-d88f-4944-8a5f-ed7544610d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:20:29.667655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 623/1262] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:20:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:20:29.689488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cd2f7-9c1b-40d3-ad0d-bebaf5c0246c None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:29.700105 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:29.790052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cd2f7-9c1b-40d3-ad0d-bebaf5c0246c None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:29.824205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424235a5-4aa5-4b98-a784-a1033d6f6dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:29.824982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424235a5-4aa5-4b98-a784-a1033d6f6dbb None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:29.825293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424235a5-4aa5-4b98-a784-a1033d6f6dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:29.825580 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424235a5-4aa5-4b98-a784-a1033d6f6dbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:29.836019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424235a5-4aa5-4b98-a784-a1033d6f6dbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-424235a5-4aa5-4b98-a784-a1033d6f6dbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:20:29.843617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:29.844711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:29.845806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:29.845806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:29.845806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:29.845806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:29.845806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:20:29.845806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:20:29.845806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:20:29.845806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:20:29.845806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:20:29.845806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:20:29.845806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:20:29.845806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:29.845806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 640/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:20:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:29.850838 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15b221-d88f-4944-8a5f-ed7544610d94 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:29.868759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:29.956115 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15b221-d88f-4944-8a5f-ed7544610d94 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:29.997106 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793b701-fe10-4b77-b227-9cec63b00046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:29.997976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793b701-fe10-4b77-b227-9cec63b00046 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:29.998259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793b701-fe10-4b77-b227-9cec63b00046 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:29.998579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793b701-fe10-4b77-b227-9cec63b00046 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:29.999432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793b701-fe10-4b77-b227-9cec63b00046 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:29.999786 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793b701-fe10-4b77-b227-9cec63b00046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:30.001494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793b701-fe10-4b77-b227-9cec63b00046 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:30.012507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 624/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:20:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:30.019393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424235a5-4aa5-4b98-a784-a1033d6f6dbb None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:30.091554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:30.124493 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424235a5-4aa5-4b98-a784-a1033d6f6dbb None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:30.157878 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550e43e-7926-4f0f-ab4f-8786c2d6a35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:30.158661 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550e43e-7926-4f0f-ab4f-8786c2d6a35b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:30.158945 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550e43e-7926-4f0f-ab4f-8786c2d6a35b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:30.159220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550e43e-7926-4f0f-ab4f-8786c2d6a35b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:30.169604 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550e43e-7926-4f0f-ab4f-8786c2d6a35b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:30.171062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550e43e-7926-4f0f-ab4f-8786c2d6a35b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:30.188228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-9550e43e-7926-4f0f-ab4f-8786c2d6a35b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:30.188228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:30.188228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:30.188228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:30.188228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:30.188228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:30.188228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:30.188228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:30.188228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:30.188228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:30.188228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:30.188228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:30.188228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:20:30.188228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:20:30.188228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:30.189096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:30.190160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:20:30.190160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:20:30.190160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:30.190160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:30.190160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 641/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:30.195861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793b701-fe10-4b77-b227-9cec63b00046 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:30.277891 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:30.297756 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793b701-fe10-4b77-b227-9cec63b00046 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:30.331529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3977c8-0805-452c-9e5d-c33ebc853d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:30.332195 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3977c8-0805-452c-9e5d-c33ebc853d87 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:30.332483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3977c8-0805-452c-9e5d-c33ebc853d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:30.332764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3977c8-0805-452c-9e5d-c33ebc853d87 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:30.333533 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3977c8-0805-452c-9e5d-c33ebc853d87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:30.333846 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3977c8-0805-452c-9e5d-c33ebc853d87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:30.335315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3977c8-0805-452c-9e5d-c33ebc853d87 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:30.347140 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 625/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:30.353310 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550e43e-7926-4f0f-ab4f-8786c2d6a35b None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:30.452032 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550e43e-7926-4f0f-ab4f-8786c2d6a35b None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:30.470518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:30.483930 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06ad51-b886-4d35-91a7-16a45127490b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:30.484716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06ad51-b886-4d35-91a7-16a45127490b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:30.484998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06ad51-b886-4d35-91a7-16a45127490b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:30.485306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06ad51-b886-4d35-91a7-16a45127490b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:30.493003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06ad51-b886-4d35-91a7-16a45127490b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:30.494477 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06ad51-b886-4d35-91a7-16a45127490b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-fa06ad51-b886-4d35-91a7-16a45127490b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:20:30.509037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:30.510178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:30.511560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:30.511560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:30.511560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:20:30.511560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:20:30.511560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:20:30.511560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:20:30.511560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:20:30.511560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:20:30.511560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 15:20:30.511560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:30.511560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 642/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:30.515866 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3977c8-0805-452c-9e5d-c33ebc853d87 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:30.612754 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3977c8-0805-452c-9e5d-c33ebc853d87 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:30.645807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577149d-8d11-4eab-9610-ef9ef8183c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:30.646465 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577149d-8d11-4eab-9610-ef9ef8183c95 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:30.646746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577149d-8d11-4eab-9610-ef9ef8183c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:30.647163 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577149d-8d11-4eab-9610-ef9ef8183c95 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:30.647996 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577149d-8d11-4eab-9610-ef9ef8183c95 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:30.648304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577149d-8d11-4eab-9610-ef9ef8183c95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:30.649749 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577149d-8d11-4eab-9610-ef9ef8183c95 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:30.658056 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:30.667141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 626/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Tue Jul 19 15:20:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:30.672761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06ad51-b886-4d35-91a7-16a45127490b None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:30.770313 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06ad51-b886-4d35-91a7-16a45127490b None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:30.802856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6817a1-feed-4f1f-8d56-611b102d4965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:30.804122 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6817a1-feed-4f1f-8d56-611b102d4965 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:30.804406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6817a1-feed-4f1f-8d56-611b102d4965 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:30.804678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6817a1-feed-4f1f-8d56-611b102d4965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:30.814992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6817a1-feed-4f1f-8d56-611b102d4965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:30.816422 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6817a1-feed-4f1f-8d56-611b102d4965 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:30.833207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-1c6817a1-feed-4f1f-8d56-611b102d4965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:30.833207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:30.833207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:30.833207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:30.833207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:30.833207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:30.833207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:30.833207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:30.833207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:30.833207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:30.833207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:30.833207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:30.833207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:20:30.833207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:20:30.833207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:30.834244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:30.835431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:20:30.835431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:20:30.835431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:30.835431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:30.835431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 643/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:30.840199 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577149d-8d11-4eab-9610-ef9ef8183c95 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:30.940699 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577149d-8d11-4eab-9610-ef9ef8183c95 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:30.977148 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793fba8-e6f4-4f27-bbb0-3736f1ab359d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:30.977905 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793fba8-e6f4-4f27-bbb0-3736f1ab359d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:30.978350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793fba8-e6f4-4f27-bbb0-3736f1ab359d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:30.978564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793fba8-e6f4-4f27-bbb0-3736f1ab359d None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:30.979324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793fba8-e6f4-4f27-bbb0-3736f1ab359d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:30.979669 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793fba8-e6f4-4f27-bbb0-3736f1ab359d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:30.981245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793fba8-e6f4-4f27-bbb0-3736f1ab359d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:30.993947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 627/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:30.999582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6817a1-feed-4f1f-8d56-611b102d4965 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:31.031158 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:31.097802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6817a1-feed-4f1f-8d56-611b102d4965 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:31.130261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ca86b-acb8-4bff-a833-6f0bfb095c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:31.131075 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ca86b-acb8-4bff-a833-6f0bfb095c1f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:31.131515 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ca86b-acb8-4bff-a833-6f0bfb095c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:31.131799 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ca86b-acb8-4bff-a833-6f0bfb095c1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:31.138328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ca86b-acb8-4bff-a833-6f0bfb095c1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:31.139764 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ca86b-acb8-4bff-a833-6f0bfb095c1f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:31.154591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-7f8ca86b-acb8-4bff-a833-6f0bfb095c1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:20:31.154591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:31.154591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:31.154591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:31.154591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:31.154591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:31.154591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:31.154591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:31.154591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:31.154591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:31.154591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:31.154591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:31.154591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:20:31.154591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:20:31.154591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:31.155634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:31.156947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:20:31.156947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:20:31.156947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:20:31.156947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:20:31.156947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:20:31.156947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:20:31.156947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:20:31.156947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:31.156947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 644/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:20:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:31.161495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793fba8-e6f4-4f27-bbb0-3736f1ab359d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:31.219419 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:31.261627 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793fba8-e6f4-4f27-bbb0-3736f1ab359d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:31.295842 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec096b-b577-40d6-b2ba-be213cb004a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:31.296524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec096b-b577-40d6-b2ba-be213cb004a5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:31.296810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec096b-b577-40d6-b2ba-be213cb004a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:31.297089 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec096b-b577-40d6-b2ba-be213cb004a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:31.297929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec096b-b577-40d6-b2ba-be213cb004a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:31.298240 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec096b-b577-40d6-b2ba-be213cb004a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:31.299683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec096b-b577-40d6-b2ba-be213cb004a5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:31.311840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 628/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:20:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:31.317763 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ca86b-acb8-4bff-a833-6f0bfb095c1f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:31.406191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:31.415077 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ca86b-acb8-4bff-a833-6f0bfb095c1f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:31.447396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a47938-c335-44b1-86af-69363025334d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:31.448255 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a47938-c335-44b1-86af-69363025334d None admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:31.448536 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a47938-c335-44b1-86af-69363025334d None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:31.448807 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a47938-c335-44b1-86af-69363025334d None admin] PATH_INFO: `/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/6002f32936944b0db6fd04f3851836db/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:31.450759 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a47938-c335-44b1-86af-69363025334d None admin] RBAC: Authorizing `identity:create_grant(project_id=a5f51e47b8e34bf5ac822d8c93853e30, user_id=6002f32936944b0db6fd04f3851836db, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:31.452181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a47938-c335-44b1-86af-69363025334d None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:31.465483 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 645/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Jul 19 15:20:31 2022] PUT /identity/v3/projects/a5f51e47b8e34bf5ac822d8c93853e30/users/6002f32936944b0db6fd04f3851836db/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:20:33.467691 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f2d7f-0b38-4e0e-9a11-d4501a393dbd None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:33.468269 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f2d7f-0b38-4e0e-9a11-d4501a393dbd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:33.468269 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f2d7f-0b38-4e0e-9a11-d4501a393dbd None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:33.469646 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 629/1274] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:20:33.474948 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e5d4b-3c71-488e-adc4-aed1a521aa27 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:33.475366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e5d4b-3c71-488e-adc4-aed1a521aa27 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:33.475568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e5d4b-3c71-488e-adc4-aed1a521aa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:33.492696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-393e5d4b-3c71-488e-adc4-aed1a521aa27 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:20:33.493936 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-393e5d4b-3c71-488e-adc4-aed1a521aa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:20:33.560713 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 646/1275] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:20:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:20:33.584209 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f2d7f-0b38-4e0e-9a11-d4501a393dbd None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:33.682507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f2d7f-0b38-4e0e-9a11-d4501a393dbd None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:33.698805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:33.715135 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb6ecc-6753-49f7-81d8-870720c4d2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:33.715908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb6ecc-6753-49f7-81d8-870720c4d2bd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:33.716187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb6ecc-6753-49f7-81d8-870720c4d2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:33.716456 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb6ecc-6753-49f7-81d8-870720c4d2bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:33.726912 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb6ecc-6753-49f7-81d8-870720c4d2bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-1beb6ecc-6753-49f7-81d8-870720c4d2bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 15:20:33.734359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:33.735545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:33.736676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:33.736676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:33.736676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:33.736676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:33.736676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 15:20:33.736676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 15:20:33.736676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 15:20:33.736676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 15:20:33.736676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 15:20:33.736676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 15:20:33.736676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 15:20:33.736676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:33.736676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 630/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Jul 19 15:20:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:33.744251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e5d4b-3c71-488e-adc4-aed1a521aa27 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:33.823938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:33.843012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e5d4b-3c71-488e-adc4-aed1a521aa27 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:33.876153 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8612e2d-c9b7-47b6-9283-f064e87a314f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:33.876813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8612e2d-c9b7-47b6-9283-f064e87a314f None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:33.877291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8612e2d-c9b7-47b6-9283-f064e87a314f None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:33.877573 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8612e2d-c9b7-47b6-9283-f064e87a314f None admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:33.878330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8612e2d-c9b7-47b6-9283-f064e87a314f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:33.878640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8612e2d-c9b7-47b6-9283-f064e87a314f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:33.880012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8612e2d-c9b7-47b6-9283-f064e87a314f None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:33.887852 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 647/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Jul 19 15:20:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:33.893454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb6ecc-6753-49f7-81d8-870720c4d2bd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:33.998766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb6ecc-6753-49f7-81d8-870720c4d2bd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:34.032483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c842f-3e59-468d-896b-cdb63a524188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:34.033300 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c842f-3e59-468d-896b-cdb63a524188 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:34.033589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c842f-3e59-468d-896b-cdb63a524188 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:34.033869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c842f-3e59-468d-896b-cdb63a524188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:34.044924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c842f-3e59-468d-896b-cdb63a524188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:34.046410 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c842f-3e59-468d-896b-cdb63a524188 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:34.063178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-805c842f-3e59-468d-896b-cdb63a524188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:34.063178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:34.063178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:34.063178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:34.063178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:34.063178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:34.063178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:34.063178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:34.063178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:34.063178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:34.063178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:34.063178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:34.063178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:20:34.063178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:20:34.063178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:34.064196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:34.065306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:20:34.065306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:20:34.065306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:34.065306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:34.065306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 631/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:34.070194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8612e2d-c9b7-47b6-9283-f064e87a314f None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:34.168602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8612e2d-c9b7-47b6-9283-f064e87a314f None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:34.201121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c36c34-de78-4413-bb8c-cf5d6482c4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:34.201791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c36c34-de78-4413-bb8c-cf5d6482c4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:34.202071 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c36c34-de78-4413-bb8c-cf5d6482c4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:34.202345 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c36c34-de78-4413-bb8c-cf5d6482c4b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:34.203059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c36c34-de78-4413-bb8c-cf5d6482c4b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:34.203364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c36c34-de78-4413-bb8c-cf5d6482c4b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:34.204724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c36c34-de78-4413-bb8c-cf5d6482c4b4 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:34.216689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 648/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:34.222161 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c842f-3e59-468d-896b-cdb63a524188 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:34.253223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:34.322799 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c842f-3e59-468d-896b-cdb63a524188 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:34.355825 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfc4aa-48f6-4507-8bcb-85a8a41c047a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:34.356627 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfc4aa-48f6-4507-8bcb-85a8a41c047a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:34.356914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfc4aa-48f6-4507-8bcb-85a8a41c047a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:34.357219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfc4aa-48f6-4507-8bcb-85a8a41c047a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:34.364468 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfc4aa-48f6-4507-8bcb-85a8a41c047a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:34.366163 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfc4aa-48f6-4507-8bcb-85a8a41c047a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-70bfc4aa-48f6-4507-8bcb-85a8a41c047a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 15:20:34.380158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:34.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:34.382743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:34.382743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:34.382743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 15:20:34.382743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 15:20:34.382743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 15:20:34.382743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 15:20:34.382743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 15:20:34.382743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 15:20:34.382743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 15:20:34.382743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:34.382743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 632/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:34.387698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c36c34-de78-4413-bb8c-cf5d6482c4b4 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:34.391506 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:34.484363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c36c34-de78-4413-bb8c-cf5d6482c4b4 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:34.517326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d30de-1abe-4349-ac1e-e3bcc146340b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:34.517972 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d30de-1abe-4349-ac1e-e3bcc146340b None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:34.518252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d30de-1abe-4349-ac1e-e3bcc146340b None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:34.518527 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d30de-1abe-4349-ac1e-e3bcc146340b None admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:34.519619 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d30de-1abe-4349-ac1e-e3bcc146340b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:34.519924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d30de-1abe-4349-ac1e-e3bcc146340b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:34.521994 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d30de-1abe-4349-ac1e-e3bcc146340b None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:34.538210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 649/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Tue Jul 19 15:20:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:34.543818 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfc4aa-48f6-4507-8bcb-85a8a41c047a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:34.633227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:34.643229 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfc4aa-48f6-4507-8bcb-85a8a41c047a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:34.675794 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c785ed-ff37-4984-a90b-7bb097049b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:34.676437 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c785ed-ff37-4984-a90b-7bb097049b6f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:34.676722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c785ed-ff37-4984-a90b-7bb097049b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:34.676999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c785ed-ff37-4984-a90b-7bb097049b6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:34.742841 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c785ed-ff37-4984-a90b-7bb097049b6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:34.744232 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c785ed-ff37-4984-a90b-7bb097049b6f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:34.828640 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:34.839508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 633/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Jul 19 15:20:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 15:20:34.845282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d30de-1abe-4349-ac1e-e3bcc146340b None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:34.904446 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:34.942207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d30de-1abe-4349-ac1e-e3bcc146340b None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:34.979949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f4345-f530-4f83-9256-f3c6834fb826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:34.980772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f4345-f530-4f83-9256-f3c6834fb826 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:34.981060 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f4345-f530-4f83-9256-f3c6834fb826 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:34.981365 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f4345-f530-4f83-9256-f3c6834fb826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:34.993272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f4345-f530-4f83-9256-f3c6834fb826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:34.994697 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f4345-f530-4f83-9256-f3c6834fb826 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:35.012094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-d44f4345-f530-4f83-9256-f3c6834fb826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:35.012094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:35.012094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:35.012094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:35.012094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:35.012094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:35.012094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:35.012094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:35.012094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:35.012094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:35.012094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:35.012094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:35.012094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 15:20:35.012094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 15:20:35.012094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:35.013012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:35.013975 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 15:20:35.013975 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 15:20:35.013975 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 15:20:35.013975 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:35.013975 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 650/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:35.019277 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c785ed-ff37-4984-a90b-7bb097049b6f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:35.116954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c785ed-ff37-4984-a90b-7bb097049b6f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:35.151630 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70bff69-a343-4305-97c8-fdcfee987d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:35.152283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bff69-a343-4305-97c8-fdcfee987d4a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:35.152567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bff69-a343-4305-97c8-fdcfee987d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:35.152849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bff69-a343-4305-97c8-fdcfee987d4a None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:35.153596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70bff69-a343-4305-97c8-fdcfee987d4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:35.153906 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70bff69-a343-4305-97c8-fdcfee987d4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:35.155275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70bff69-a343-4305-97c8-fdcfee987d4a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:35.166168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 634/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Jul 19 15:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:35.171686 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f4345-f530-4f83-9256-f3c6834fb826 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:35.276548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f4345-f530-4f83-9256-f3c6834fb826 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:35.290801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:35.310640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8addb-43c6-4158-a245-753e696a72da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:35.311429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8addb-43c6-4158-a245-753e696a72da None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:35.311716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8addb-43c6-4158-a245-753e696a72da None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:35.311994 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8addb-43c6-4158-a245-753e696a72da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:35.319054 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8addb-43c6-4158-a245-753e696a72da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:35.320509 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8addb-43c6-4158-a245-753e696a72da None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:35.334584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application [None req-b0a8addb-43c6-4158-a245-753e696a72da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:20:35.334584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:35.334584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:35.334584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:35.334584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:35.334584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:35.334584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:35.334584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:35.334584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:35.334584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:35.334584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:35.334584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:35.334584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 15:20:35.334584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 15:20:35.334584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:35.335671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:35.337000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 15:20:35.337000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 15:20:35.337000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 15:20:35.337000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 15:20:35.337000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 15:20:35.337000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 15:20:35.337000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 15:20:35.337000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: ERROR keystone.server.flask.application Jul 19 15:20:35.337000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 651/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Jul 19 15:20:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:35.341763 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70bff69-a343-4305-97c8-fdcfee987d4a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:35.393257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:35.444688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70bff69-a343-4305-97c8-fdcfee987d4a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:35.478423 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd1822-8002-4f7b-9323-84138e9eaf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:35.479080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd1822-8002-4f7b-9323-84138e9eaf94 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:35.479361 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd1822-8002-4f7b-9323-84138e9eaf94 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:35.479634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd1822-8002-4f7b-9323-84138e9eaf94 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:35.480416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd1822-8002-4f7b-9323-84138e9eaf94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:35.480722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd1822-8002-4f7b-9323-84138e9eaf94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:35.482251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd1822-8002-4f7b-9323-84138e9eaf94 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:35.498714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 635/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Jul 19 15:20:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:35.504935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8addb-43c6-4158-a245-753e696a72da None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:35.607615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8addb-43c6-4158-a245-753e696a72da None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:35.641945 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a18e82-c4be-49bd-94ce-38b35e06fd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:35.642601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a18e82-c4be-49bd-94ce-38b35e06fd89 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:35.642888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a18e82-c4be-49bd-94ce-38b35e06fd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:35.643162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a18e82-c4be-49bd-94ce-38b35e06fd89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:35.644242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a18e82-c4be-49bd-94ce-38b35e06fd89 None admin] RBAC: Adding query filter params (role.id=113045fe7ac44a0998b8ec6fe410d564, scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30, user.id=6002f32936944b0db6fd04f3851836db) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:35.644550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a18e82-c4be-49bd-94ce-38b35e06fd89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:35.646092 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a18e82-c4be-49bd-94ce-38b35e06fd89 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:35.652178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 652/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Jul 19 15:20:35 2022] GET /identity/v3/role_assignments?user.id=6002f32936944b0db6fd04f3851836db&scope.project.id=a5f51e47b8e34bf5ac822d8c93853e30&role.id=113045fe7ac44a0998b8ec6fe410d564 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:37.720222 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce524e-0243-4e26-bde4-bb27bb95ec72 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:37.720743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce524e-0243-4e26-bde4-bb27bb95ec72 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:37.720743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce524e-0243-4e26-bde4-bb27bb95ec72 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:37.722101 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 636/1288] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:20:37.727858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04120a7d-5f14-4d71-9aec-f64a8a3d8ec2 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:37.728252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04120a7d-5f14-4d71-9aec-f64a8a3d8ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:37.728411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04120a7d-5f14-4d71-9aec-f64a8a3d8ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:37.745023 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-04120a7d-5f14-4d71-9aec-f64a8a3d8ec2 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:20:37.746231 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-04120a7d-5f14-4d71-9aec-f64a8a3d8ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:20:37.774820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:37.810442 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 653/1289] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:20:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:20:37.832257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce524e-0243-4e26-bde4-bb27bb95ec72 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:37.932267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce524e-0243-4e26-bde4-bb27bb95ec72 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:37.942599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:37.973075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6bec5-6474-4bd5-bdfa-89bb4451fefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:37.974391 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6bec5-6474-4bd5-bdfa-89bb4451fefb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:37.974391 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6bec5-6474-4bd5-bdfa-89bb4451fefb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:37.975359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6bec5-6474-4bd5-bdfa-89bb4451fefb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:37.976136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f6bec5-6474-4bd5-bdfa-89bb4451fefb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:37.977611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f6bec5-6474-4bd5-bdfa-89bb4451fefb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-80f6bec5-6474-4bd5-bdfa-89bb4451fefb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:37.996235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:37.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:37.998777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:37.998777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:37.998777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:37.998777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:37.998777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:37.998777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:37.998777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:37.998777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:37.998777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:37.998777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:37.998777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:20:37.998777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:20:37.998777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 15:20:37.998777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:37.998777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 637/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Jul 19 15:20:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:38.004457 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04120a7d-5f14-4d71-9aec-f64a8a3d8ec2 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:38.113712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04120a7d-5f14-4d71-9aec-f64a8a3d8ec2 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:38.148514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d796f-8ae5-4476-ac3b-6f6bb59826bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:38.149197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d796f-8ae5-4476-ac3b-6f6bb59826bc None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:38.149486 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d796f-8ae5-4476-ac3b-6f6bb59826bc None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:38.149765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d796f-8ae5-4476-ac3b-6f6bb59826bc None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:38.150471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d796f-8ae5-4476-ac3b-6f6bb59826bc None admin] RBAC: Adding query filter params (name=placement) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:38.150779 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d796f-8ae5-4476-ac3b-6f6bb59826bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:38.152142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d796f-8ae5-4476-ac3b-6f6bb59826bc None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:38.156851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 654/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Tue Jul 19 15:20:38 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:38.162860 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6bec5-6474-4bd5-bdfa-89bb4451fefb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:38.261761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6bec5-6474-4bd5-bdfa-89bb4451fefb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:38.299102 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e42212-a2b0-4bd2-a47e-c1f10bc85b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:38.300119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e42212-a2b0-4bd2-a47e-c1f10bc85b87 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:38.300618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e42212-a2b0-4bd2-a47e-c1f10bc85b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:38.301168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e42212-a2b0-4bd2-a47e-c1f10bc85b87 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:38.302505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e42212-a2b0-4bd2-a47e-c1f10bc85b87 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:38.303060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e42212-a2b0-4bd2-a47e-c1f10bc85b87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:38.305564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e42212-a2b0-4bd2-a47e-c1f10bc85b87 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:38.310633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 638/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Tue Jul 19 15:20:38 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:40.304413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59f618-14f4-4fe9-b612-346eb9d4ce14 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:40.304897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59f618-14f4-4fe9-b612-346eb9d4ce14 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:40.304988 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59f618-14f4-4fe9-b612-346eb9d4ce14 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:40.306423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 655/1293] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:20:40.311596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59770cee-8960-4510-bf0c-dac7532f4418 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:40.311964 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59770cee-8960-4510-bf0c-dac7532f4418 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:40.312157 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59770cee-8960-4510-bf0c-dac7532f4418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:40.329504 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-59770cee-8960-4510-bf0c-dac7532f4418 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:20:40.330715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-59770cee-8960-4510-bf0c-dac7532f4418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:20:40.360381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:40.397845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 639/1294] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:20:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:20:40.420543 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59f618-14f4-4fe9-b612-346eb9d4ce14 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:40.451298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:40.520567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59f618-14f4-4fe9-b612-346eb9d4ce14 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:40.554227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373f862-94b8-46cf-b688-46bf4f6b89ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:40.554880 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373f862-94b8-46cf-b688-46bf4f6b89ed None admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:40.555169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373f862-94b8-46cf-b688-46bf4f6b89ed None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:40.555450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373f862-94b8-46cf-b688-46bf4f6b89ed None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:40.556181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373f862-94b8-46cf-b688-46bf4f6b89ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:40.557590 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373f862-94b8-46cf-b688-46bf4f6b89ed None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:40.570306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 656/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Jul 19 15:20:40 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:20:42.611045 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940353a0-37db-4eca-b1ef-21b3e5b72e28 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:42.611540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940353a0-37db-4eca-b1ef-21b3e5b72e28 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:42.611540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940353a0-37db-4eca-b1ef-21b3e5b72e28 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:42.612890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 640/1296] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:20:42.619257 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ad133-65ad-442d-8411-e25740ef51c2 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:42.619633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ad133-65ad-442d-8411-e25740ef51c2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:42.619821 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ad133-65ad-442d-8411-e25740ef51c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:42.636692 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-962ad133-65ad-442d-8411-e25740ef51c2 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:20:42.637897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-962ad133-65ad-442d-8411-e25740ef51c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:20:42.710287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 657/1297] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:20:42 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:20:42.733867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940353a0-37db-4eca-b1ef-21b3e5b72e28 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:42.831969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940353a0-37db-4eca-b1ef-21b3e5b72e28 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:42.864159 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df239211-0e5c-4da5-a6c6-76b4487a7801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:42.865615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df239211-0e5c-4da5-a6c6-76b4487a7801 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:42.865899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df239211-0e5c-4da5-a6c6-76b4487a7801 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:42.866172 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df239211-0e5c-4da5-a6c6-76b4487a7801 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:42.866819 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df239211-0e5c-4da5-a6c6-76b4487a7801 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:42.868163 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df239211-0e5c-4da5-a6c6-76b4487a7801 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-df239211-0e5c-4da5-a6c6-76b4487a7801 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:42.879582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:42.880983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:42.881884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:42.881884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:42.881884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:42.881884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:42.881884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:42.881884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:42.881884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:42.881884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:42.881884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:42.881884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:42.881884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:20:42.881884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:20:42.881884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 15:20:42.881884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:42.881884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 641/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Jul 19 15:20:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:42.886900 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ad133-65ad-442d-8411-e25740ef51c2 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:42.905474 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:42.991717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ad133-65ad-442d-8411-e25740ef51c2 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:43.034070 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805814b2-c5a5-495e-a897-2ab5c3b01cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:43.034745 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805814b2-c5a5-495e-a897-2ab5c3b01cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:43.035032 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805814b2-c5a5-495e-a897-2ab5c3b01cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:43.035312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805814b2-c5a5-495e-a897-2ab5c3b01cd2 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:43.036030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805814b2-c5a5-495e-a897-2ab5c3b01cd2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:43.036999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805814b2-c5a5-495e-a897-2ab5c3b01cd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:43.038417 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805814b2-c5a5-495e-a897-2ab5c3b01cd2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:43.043597 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 658/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Tue Jul 19 15:20:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:20:43.049585 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df239211-0e5c-4da5-a6c6-76b4487a7801 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:43.081198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:43.149208 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df239211-0e5c-4da5-a6c6-76b4487a7801 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:43.181349 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ef7a2-8a82-4aa9-b79b-cd1dfb1383e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:43.181999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ef7a2-8a82-4aa9-b79b-cd1dfb1383e2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:43.182283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ef7a2-8a82-4aa9-b79b-cd1dfb1383e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:43.182561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ef7a2-8a82-4aa9-b79b-cd1dfb1383e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:43.183351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ef7a2-8a82-4aa9-b79b-cd1dfb1383e2 None admin] RBAC: Adding query filter params (interface=public, service_id=76690e02df844488b98d5ba34bfc6ec9, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:43.183664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ef7a2-8a82-4aa9-b79b-cd1dfb1383e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:43.185038 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ef7a2-8a82-4aa9-b79b-cd1dfb1383e2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:43.189964 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 642/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Jul 19 15:20:43 2022] GET /identity/v3/endpoints?service_id=76690e02df844488b98d5ba34bfc6ec9&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:43.195791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805814b2-c5a5-495e-a897-2ab5c3b01cd2 None admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:43.278783 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:43.296994 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805814b2-c5a5-495e-a897-2ab5c3b01cd2 None admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:43.329745 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153a7d5-3a77-42e2-a479-7ff0c7f58bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:43.330397 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153a7d5-3a77-42e2-a479-7ff0c7f58bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:43.330676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153a7d5-3a77-42e2-a479-7ff0c7f58bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:43.330949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153a7d5-3a77-42e2-a479-7ff0c7f58bc2 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:43.331631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153a7d5-3a77-42e2-a479-7ff0c7f58bc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:43.332983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153a7d5-3a77-42e2-a479-7ff0c7f58bc2 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:43.340537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 659/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Jul 19 15:20:43 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:20:45.333818 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713b8c7-e437-494f-86cd-0236dd8510bd None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:45.334330 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713b8c7-e437-494f-86cd-0236dd8510bd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:45.334441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713b8c7-e437-494f-86cd-0236dd8510bd None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:45.335759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 643/1302] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:20:45.341025 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12da07-c6e3-49a0-84ba-d1007c6db5ee None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:45.341468 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12da07-c6e3-49a0-84ba-d1007c6db5ee None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:45.341602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12da07-c6e3-49a0-84ba-d1007c6db5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:45.357890 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-2b12da07-c6e3-49a0-84ba-d1007c6db5ee None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:20:45.359056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2b12da07-c6e3-49a0-84ba-d1007c6db5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:20:45.423825 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 660/1303] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:20:45 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 15:20:45.446556 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713b8c7-e437-494f-86cd-0236dd8510bd None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:45.542294 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:45.549498 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713b8c7-e437-494f-86cd-0236dd8510bd None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:45.583059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466407d5-bef5-4aa6-817c-20c8844f712b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:45.583981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466407d5-bef5-4aa6-817c-20c8844f712b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:45.584268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466407d5-bef5-4aa6-817c-20c8844f712b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:45.584545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466407d5-bef5-4aa6-817c-20c8844f712b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:45.585223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466407d5-bef5-4aa6-817c-20c8844f712b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:45.586589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466407d5-bef5-4aa6-817c-20c8844f712b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-466407d5-bef5-4aa6-817c-20c8844f712b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 15:20:45.598066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 15:20:45.599948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 15:20:45.601254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 15:20:45.601254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 15:20:45.601254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 15:20:45.601254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 15:20:45.601254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 15:20:45.601254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 15:20:45.601254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 15:20:45.601254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 15:20:45.601254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 15:20:45.601254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 15:20:45.601254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 15:20:45.601254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 15:20:45.601254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 15:20:45.601254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 15:20:45.601254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 644/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Jul 19 15:20:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 15:20:45.605996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12da07-c6e3-49a0-84ba-d1007c6db5ee None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:45.680802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:45.703407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12da07-c6e3-49a0-84ba-d1007c6db5ee None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:45.736263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e30f8-50d7-4442-8c23-023e56ae3673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:45.736917 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e30f8-50d7-4442-8c23-023e56ae3673 None admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:45.737232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e30f8-50d7-4442-8c23-023e56ae3673 None admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:45.737514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e30f8-50d7-4442-8c23-023e56ae3673 None admin] PATH_INFO: `/v3/services` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:45.738226 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e30f8-50d7-4442-8c23-023e56ae3673 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:20:45.738537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e30f8-50d7-4442-8c23-023e56ae3673 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:45.740462 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e30f8-50d7-4442-8c23-023e56ae3673 None admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:45.745528 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 661/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Tue Jul 19 15:20:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:20:45.751085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466407d5-bef5-4aa6-817c-20c8844f712b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:20:45.850150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466407d5-bef5-4aa6-817c-20c8844f712b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:20:45.882665 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c300b0b-0077-4a6f-a29e-5e50164839c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:20:45.883319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c300b0b-0077-4a6f-a29e-5e50164839c8 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:20:45.883606 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c300b0b-0077-4a6f-a29e-5e50164839c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:20:45.883884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c300b0b-0077-4a6f-a29e-5e50164839c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:20:45.884638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c300b0b-0077-4a6f-a29e-5e50164839c8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:20:45.886034 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c300b0b-0077-4a6f-a29e-5e50164839c8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:20:45.904244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 645/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Jul 19 15:20:45 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:20:55.742516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=30 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:20:55.899989 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:21:48.842694 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9104a1-811d-42af-a480-75cbb0a1dd9b None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:48.843240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9104a1-811d-42af-a480-75cbb0a1dd9b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:48.843240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9104a1-811d-42af-a480-75cbb0a1dd9b None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:48.851089 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692e5ff-ec4b-413a-80cf-ca757860eb91 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:48.851611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692e5ff-ec4b-413a-80cf-ca757860eb91 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:48.851611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692e5ff-ec4b-413a-80cf-ca757860eb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:48.854918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 662/1307] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:21:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:21:48.942450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4692e5ff-ec4b-413a-80cf-ca757860eb91 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:21:48.943917 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4692e5ff-ec4b-413a-80cf-ca757860eb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:21:49.068448 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 646/1308] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:21:49.142500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2514ef-c325-468c-986a-5f87781697a6 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:49.143143 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2514ef-c325-468c-986a-5f87781697a6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:49.143664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2514ef-c325-468c-986a-5f87781697a6 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:49.145606 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 663/1309] 10.222.0.42 () {54 vars in 965 bytes} [Tue Jul 19 15:21:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:21:49.151971 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec6221-ba99-47cc-b407-26c5846685a5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:49.152255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec6221-ba99-47cc-b407-26c5846685a5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:49.152540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec6221-ba99-47cc-b407-26c5846685a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:49.254352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-cdec6221-ba99-47cc-b407-26c5846685a5 None None] MFA Rules not processed for user `369bdaded0534142b03ad3cf43a70881`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:21:49.256757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cdec6221-ba99-47cc-b407-26c5846685a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:21:49.293817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:21:49.433398 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 647/1310] 10.222.0.42 () {58 vars in 1062 bytes} [Tue Jul 19 15:21:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:21:49.443157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2514ef-c325-468c-986a-5f87781697a6 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:21:49.444898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7c2514ef-c325-468c-986a-5f87781697a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:21:49.590519 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2514ef-c325-468c-986a-5f87781697a6 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:21:49.670033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5541f-eb18-4f61-a182-ecb53d0c593c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:21:49.670612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5541f-eb18-4f61-a182-ecb53d0c593c service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:49.670896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5541f-eb18-4f61-a182-ecb53d0c593c service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:49.671176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5541f-eb18-4f61-a182-ecb53d0c593c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:49.673538 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-66a5541f-eb18-4f61-a182-ecb53d0c593c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:21:49.736730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:21:49.800096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5541f-eb18-4f61-a182-ecb53d0c593c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:21:49.801624 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5541f-eb18-4f61-a182-ecb53d0c593c service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:21:49.823649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 664/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:21:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:21:52.210705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2dcec-872c-4558-a70f-2b0ee84e6ffb None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:52.211248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2dcec-872c-4558-a70f-2b0ee84e6ffb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:52.211367 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2dcec-872c-4558-a70f-2b0ee84e6ffb None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:52.212673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 648/1312] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:21:52.217924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4c129-f8f5-48fb-88af-8fa8c7f7e75b None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:52.218314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4c129-f8f5-48fb-88af-8fa8c7f7e75b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:52.218492 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4c129-f8f5-48fb-88af-8fa8c7f7e75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:52.291259 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-0bf4c129-f8f5-48fb-88af-8fa8c7f7e75b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:21:52.292468 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0bf4c129-f8f5-48fb-88af-8fa8c7f7e75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:21:52.390879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 665/1313] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:21:52.443944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8607fbe-824d-49d9-903a-1149790d56a7 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:52.444240 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8607fbe-824d-49d9-903a-1149790d56a7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:52.444521 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8607fbe-824d-49d9-903a-1149790d56a7 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:52.445944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 649/1314] 10.222.0.42 () {54 vars in 965 bytes} [Tue Jul 19 15:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:21:52.452227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f9302-e9dd-4f33-a123-c843a36a9cbc None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:52.452520 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f9302-e9dd-4f33-a123-c843a36a9cbc None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:52.452804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f9302-e9dd-4f33-a123-c843a36a9cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:52.465212 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:21:52.538368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-956f9302-e9dd-4f33-a123-c843a36a9cbc None None] MFA Rules not processed for user `369bdaded0534142b03ad3cf43a70881`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:21:52.539658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-956f9302-e9dd-4f33-a123-c843a36a9cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:21:52.624053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 666/1315] 10.222.0.42 () {58 vars in 1062 bytes} [Tue Jul 19 15:21:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:21:52.633874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8607fbe-824d-49d9-903a-1149790d56a7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:21:52.635396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a8607fbe-824d-49d9-903a-1149790d56a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:21:52.745042 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8607fbe-824d-49d9-903a-1149790d56a7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:21:52.819676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:21:52.822453 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435fc71-3e69-4f63-be06-a54617b1eed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:21:52.823287 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435fc71-3e69-4f63-be06-a54617b1eed3 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:52.823735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435fc71-3e69-4f63-be06-a54617b1eed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:52.824201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435fc71-3e69-4f63-be06-a54617b1eed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:52.825914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2435fc71-3e69-4f63-be06-a54617b1eed3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:21:52.934837 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435fc71-3e69-4f63-be06-a54617b1eed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:21:52.936287 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435fc71-3e69-4f63-be06-a54617b1eed3 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:21:52.955434 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 650/1316] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:21:55.472154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f28d4-34dc-4df8-a6a0-04b7ba0065ee None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:55.472683 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f28d4-34dc-4df8-a6a0-04b7ba0065ee None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:55.472683 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f28d4-34dc-4df8-a6a0-04b7ba0065ee None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:55.474127 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 667/1317] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:21:55.479261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc47100-1d7f-4224-9cff-5a420d6323f8 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:55.479733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc47100-1d7f-4224-9cff-5a420d6323f8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:55.479826 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc47100-1d7f-4224-9cff-5a420d6323f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:55.544522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4bc47100-1d7f-4224-9cff-5a420d6323f8 None None] MFA Rules not processed for user `285435c09061437080caaff56db1dced`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:21:55.545680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4bc47100-1d7f-4224-9cff-5a420d6323f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:21:55.597039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:21:55.641970 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 651/1318] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:21:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:21:55.683214 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f28d4-34dc-4df8-a6a0-04b7ba0065ee None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:21:55.684752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1b5f28d4-34dc-4df8-a6a0-04b7ba0065ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:21:55.788137 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f28d4-34dc-4df8-a6a0-04b7ba0065ee None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:21:55.791030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:21:55.861075 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa147d-4256-4e59-a948-b3a98732368e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:21:55.861687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa147d-4256-4e59-a948-b3a98732368e service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:55.861972 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa147d-4256-4e59-a948-b3a98732368e service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:55.862256 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa147d-4256-4e59-a948-b3a98732368e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:55.864168 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-28fa147d-4256-4e59-a948-b3a98732368e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:21:55.976047 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa147d-4256-4e59-a948-b3a98732368e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:21:55.977744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa147d-4256-4e59-a948-b3a98732368e service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:21:55.998653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 668/1319] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:21:55 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:21:58.717974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121933a-bf08-49d8-937c-27c174184cd0 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:58.719512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121933a-bf08-49d8-937c-27c174184cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:58.719512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121933a-bf08-49d8-937c-27c174184cd0 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:58.720390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 652/1320] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:21:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:21:58.725651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff3c91-965e-4bdd-90be-4c3d429e2d69 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:58.726042 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff3c91-965e-4bdd-90be-4c3d429e2d69 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:58.726202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff3c91-965e-4bdd-90be-4c3d429e2d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:58.764990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:21:58.790195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-b5ff3c91-965e-4bdd-90be-4c3d429e2d69 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:21:58.791393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b5ff3c91-965e-4bdd-90be-4c3d429e2d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:21:58.885693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 669/1321] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:21:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:21:58.924937 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121933a-bf08-49d8-937c-27c174184cd0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:21:58.926516 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4121933a-bf08-49d8-937c-27c174184cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:21:59.033556 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121933a-bf08-49d8-937c-27c174184cd0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:21:59.105215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293287b-9f3e-4bb2-bbe3-879abbc58f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:21:59.105856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293287b-9f3e-4bb2-bbe3-879abbc58f93 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:21:59.106134 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293287b-9f3e-4bb2-bbe3-879abbc58f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:21:59.106410 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293287b-9f3e-4bb2-bbe3-879abbc58f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:21:59.108046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8293287b-9f3e-4bb2-bbe3-879abbc58f93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:21:59.116164 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:21:59.216560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293287b-9f3e-4bb2-bbe3-879abbc58f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:21:59.218039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293287b-9f3e-4bb2-bbe3-879abbc58f93 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:21:59.238904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 653/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:21:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:22:02.005020 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f772e3e-4d3b-4691-b2f6-f7527dbf4e3e None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:02.005613 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f772e3e-4d3b-4691-b2f6-f7527dbf4e3e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:02.005712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f772e3e-4d3b-4691-b2f6-f7527dbf4e3e None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:02.007091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 670/1323] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:02.012469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40f896-a170-4b88-b474-8cd040b38983 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:02.012908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40f896-a170-4b88-b474-8cd040b38983 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:02.013162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40f896-a170-4b88-b474-8cd040b38983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:02.084140 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bd40f896-a170-4b88-b474-8cd040b38983 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:02.085692 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bd40f896-a170-4b88-b474-8cd040b38983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:02.151922 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:02.196335 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 654/1324] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:02.235903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f772e3e-4d3b-4691-b2f6-f7527dbf4e3e None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:02.237569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9f772e3e-4d3b-4691-b2f6-f7527dbf4e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:02.353917 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f772e3e-4d3b-4691-b2f6-f7527dbf4e3e None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:02.366230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:02.428073 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b9fe4-934d-423b-a5c1-1e0f51ccd7d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:02.428725 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b9fe4-934d-423b-a5c1-1e0f51ccd7d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:02.429007 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b9fe4-934d-423b-a5c1-1e0f51ccd7d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:02.429319 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b9fe4-934d-423b-a5c1-1e0f51ccd7d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:02.430971 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-4b8b9fe4-934d-423b-a5c1-1e0f51ccd7d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:02.539128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b9fe4-934d-423b-a5c1-1e0f51ccd7d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:02.540472 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b9fe4-934d-423b-a5c1-1e0f51ccd7d9 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:02.559621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 671/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:22:04.659184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d5f92-2245-433b-9eea-8c5523c1ee43 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:04.659785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d5f92-2245-433b-9eea-8c5523c1ee43 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:04.659785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d5f92-2245-433b-9eea-8c5523c1ee43 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:04.661001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 655/1326] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:04.666255 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c0b4a-d014-44e5-90e9-af546fd76907 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:04.666697 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c0b4a-d014-44e5-90e9-af546fd76907 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:04.666939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c0b4a-d014-44e5-90e9-af546fd76907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:04.713801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:04.736791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-117c0b4a-d014-44e5-90e9-af546fd76907 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:04.738913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-117c0b4a-d014-44e5-90e9-af546fd76907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:04.840014 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 672/1327] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:04.876686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d5f92-2245-433b-9eea-8c5523c1ee43 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:04.878262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4e6d5f92-2245-433b-9eea-8c5523c1ee43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:04.987539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d5f92-2245-433b-9eea-8c5523c1ee43 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:05.070368 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e6e35-7c66-472e-8c1f-6f81cd5e9ae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:05.071039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e6e35-7c66-472e-8c1f-6f81cd5e9ae2 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:05.071325 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e6e35-7c66-472e-8c1f-6f81cd5e9ae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:05.071599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e6e35-7c66-472e-8c1f-6f81cd5e9ae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:05.073373 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-583e6e35-7c66-472e-8c1f-6f81cd5e9ae2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:05.092865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:05.188087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e6e35-7c66-472e-8c1f-6f81cd5e9ae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:05.189566 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e6e35-7c66-472e-8c1f-6f81cd5e9ae2 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:05.208204 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 656/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:22:07.291923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f901e0-dfd1-4112-a1fe-c3d267d64ee7 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:07.292459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f901e0-dfd1-4112-a1fe-c3d267d64ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:07.292459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f901e0-dfd1-4112-a1fe-c3d267d64ee7 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:07.293872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 673/1329] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:07.299660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2201c-0852-405b-974a-58556fbc7e75 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:07.300068 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2201c-0852-405b-974a-58556fbc7e75 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:07.300232 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2201c-0852-405b-974a-58556fbc7e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:07.367925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c6a2201c-0852-405b-974a-58556fbc7e75 None None] MFA Rules not processed for user `285435c09061437080caaff56db1dced`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:07.369063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c6a2201c-0852-405b-974a-58556fbc7e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:07.436677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:07.472268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 657/1330] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:07 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:07.510271 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f901e0-dfd1-4112-a1fe-c3d267d64ee7 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:07.511778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-49f901e0-dfd1-4112-a1fe-c3d267d64ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:07.618774 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f901e0-dfd1-4112-a1fe-c3d267d64ee7 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:07.638799 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:07.694618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094995d-ac6b-4c79-b1e6-0f8da114428d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:07.695284 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094995d-ac6b-4c79-b1e6-0f8da114428d service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:07.695561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094995d-ac6b-4c79-b1e6-0f8da114428d service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:07.695839 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094995d-ac6b-4c79-b1e6-0f8da114428d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:07.697559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5094995d-ac6b-4c79-b1e6-0f8da114428d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:07.807102 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094995d-ac6b-4c79-b1e6-0f8da114428d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:07.808533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094995d-ac6b-4c79-b1e6-0f8da114428d service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:07.827764 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 674/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:07 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:22:10.171476 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae708772-92a1-4cce-82b9-83d2630309a8 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:10.172012 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae708772-92a1-4cce-82b9-83d2630309a8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:10.172206 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae708772-92a1-4cce-82b9-83d2630309a8 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:10.173931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 658/1332] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:10.179217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461fa57-b0c0-454d-bb3e-731293a2da04 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:10.179622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461fa57-b0c0-454d-bb3e-731293a2da04 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:10.179763 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461fa57-b0c0-454d-bb3e-731293a2da04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:10.240587 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:10.243917 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-0461fa57-b0c0-454d-bb3e-731293a2da04 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:10.245658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0461fa57-b0c0-454d-bb3e-731293a2da04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:10.340081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 675/1333] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:10.376782 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae708772-92a1-4cce-82b9-83d2630309a8 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:10.378372 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ae708772-92a1-4cce-82b9-83d2630309a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:10.482144 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae708772-92a1-4cce-82b9-83d2630309a8 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:10.554248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f05b3d-5780-4895-812e-0e24829bdce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:10.554893 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f05b3d-5780-4895-812e-0e24829bdce2 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:10.555171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f05b3d-5780-4895-812e-0e24829bdce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:10.555442 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f05b3d-5780-4895-812e-0e24829bdce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:10.557062 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c2f05b3d-5780-4895-812e-0e24829bdce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:10.583858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:10.664854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f05b3d-5780-4895-812e-0e24829bdce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:10.666316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f05b3d-5780-4895-812e-0e24829bdce2 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:10.683917 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 659/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:22:12.808404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331f4ff-f994-4116-901e-98fd322c0724 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:12.808798 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331f4ff-f994-4116-901e-98fd322c0724 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:12.808954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331f4ff-f994-4116-901e-98fd322c0724 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:12.810375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 676/1335] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:12.815636 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1062d54-4c00-4099-b345-4e23c2b7eaab None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:12.816080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1062d54-4c00-4099-b345-4e23c2b7eaab None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:12.816177 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1062d54-4c00-4099-b345-4e23c2b7eaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:12.880194 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f1062d54-4c00-4099-b345-4e23c2b7eaab None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:12.881440 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f1062d54-4c00-4099-b345-4e23c2b7eaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:12.954673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:12.978513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 660/1336] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:13.019658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331f4ff-f994-4116-901e-98fd322c0724 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:13.021206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-8331f4ff-f994-4116-901e-98fd322c0724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:13.124127 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331f4ff-f994-4116-901e-98fd322c0724 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:13.151844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:13.196489 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28cfaa9-9ac2-4b4c-b0e1-f7bfdb4bac86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:13.197129 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cfaa9-9ac2-4b4c-b0e1-f7bfdb4bac86 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:13.197435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cfaa9-9ac2-4b4c-b0e1-f7bfdb4bac86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:13.197709 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cfaa9-9ac2-4b4c-b0e1-f7bfdb4bac86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:13.199324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c28cfaa9-9ac2-4b4c-b0e1-f7bfdb4bac86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:13.307920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28cfaa9-9ac2-4b4c-b0e1-f7bfdb4bac86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:13.309389 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28cfaa9-9ac2-4b4c-b0e1-f7bfdb4bac86 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:13.327902 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 677/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:22:15.696725 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34c06a-8da9-44b7-a508-e0545ac5d740 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:15.697263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34c06a-8da9-44b7-a508-e0545ac5d740 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:15.697378 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34c06a-8da9-44b7-a508-e0545ac5d740 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:15.698679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 661/1338] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:15.703757 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c70973-eb61-4cfa-b733-e236be2bda62 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:15.704162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c70973-eb61-4cfa-b733-e236be2bda62 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:15.704312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c70973-eb61-4cfa-b733-e236be2bda62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:15.767761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-b7c70973-eb61-4cfa-b733-e236be2bda62 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:15.768955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b7c70973-eb61-4cfa-b733-e236be2bda62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:15.777089 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:15.866240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 678/1339] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:15.902362 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34c06a-8da9-44b7-a508-e0545ac5d740 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:15.903948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ff34c06a-8da9-44b7-a508-e0545ac5d740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:16.010615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34c06a-8da9-44b7-a508-e0545ac5d740 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:16.083164 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a2b7c-d620-499a-afb1-b6118a286f9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:16.083814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a2b7c-d620-499a-afb1-b6118a286f9e service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:16.084100 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a2b7c-d620-499a-afb1-b6118a286f9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:16.084378 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a2b7c-d620-499a-afb1-b6118a286f9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:16.086057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c54a2b7c-d620-499a-afb1-b6118a286f9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:16.121127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:16.196291 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a2b7c-d620-499a-afb1-b6118a286f9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:16.197676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a2b7c-d620-499a-afb1-b6118a286f9e service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:16.216520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 662/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:22:20.179104 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa6085-593c-466d-942a-bb163459a2c8 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:20.179650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa6085-593c-466d-942a-bb163459a2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:20.179650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa6085-593c-466d-942a-bb163459a2c8 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:20.181027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 679/1341] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:20.186679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6e041-7cc7-4b3b-851d-a049a2eadfc8 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:20.187121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6e041-7cc7-4b3b-851d-a049a2eadfc8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:20.187239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6e041-7cc7-4b3b-851d-a049a2eadfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:20.259937 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0ee6e041-7cc7-4b3b-851d-a049a2eadfc8 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:20.261161 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0ee6e041-7cc7-4b3b-851d-a049a2eadfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:20.348621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:20.360706 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 663/1342] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:20.401721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa6085-593c-466d-942a-bb163459a2c8 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:20.403530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-58aa6085-593c-466d-942a-bb163459a2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:20.512668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa6085-593c-466d-942a-bb163459a2c8 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:20.551747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:20.588200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49dfce9-597b-4e97-827f-22d4281ee629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:20.588854 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dfce9-597b-4e97-827f-22d4281ee629 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:20.589153 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dfce9-597b-4e97-827f-22d4281ee629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:20.589441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dfce9-597b-4e97-827f-22d4281ee629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:20.591089 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d49dfce9-597b-4e97-827f-22d4281ee629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:20.703996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49dfce9-597b-4e97-827f-22d4281ee629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:20.705393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49dfce9-597b-4e97-827f-22d4281ee629 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:20.723377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 680/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:22:23.546683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cf745-6db3-49a3-823b-c75584420b6a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:23.547212 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cf745-6db3-49a3-823b-c75584420b6a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:23.547212 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cf745-6db3-49a3-823b-c75584420b6a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:23.548521 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 664/1344] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:23.553803 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f9acd-49fe-4fae-a456-1438406a3619 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:23.554234 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f9acd-49fe-4fae-a456-1438406a3619 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:23.554481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f9acd-49fe-4fae-a456-1438406a3619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:23.618531 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-1c9f9acd-49fe-4fae-a456-1438406a3619 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:23.619726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1c9f9acd-49fe-4fae-a456-1438406a3619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:23.637905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:23.717121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 681/1345] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:23.755870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cf745-6db3-49a3-823b-c75584420b6a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:23.757511 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ef0cf745-6db3-49a3-823b-c75584420b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:23.862616 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cf745-6db3-49a3-823b-c75584420b6a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:23.936763 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3f27a-5980-419d-8c65-8c37cb7eacfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:23.937438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3f27a-5980-419d-8c65-8c37cb7eacfc service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:23.937726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3f27a-5980-419d-8c65-8c37cb7eacfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:23.938006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3f27a-5980-419d-8c65-8c37cb7eacfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:23.939652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d6b3f27a-5980-419d-8c65-8c37cb7eacfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:23.986820 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:24.059351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3f27a-5980-419d-8c65-8c37cb7eacfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:24.060951 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3f27a-5980-419d-8c65-8c37cb7eacfc service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:24.079519 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 665/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:22:28.316172 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302cef8-dddc-40d3-9eaf-fccbf5724041 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:28.316736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302cef8-dddc-40d3-9eaf-fccbf5724041 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:28.316736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302cef8-dddc-40d3-9eaf-fccbf5724041 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:28.318131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 682/1347] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:28.323673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f2f68-51bf-4b8d-b6de-888e36f0c642 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:28.324098 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f2f68-51bf-4b8d-b6de-888e36f0c642 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:28.324309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f2f68-51bf-4b8d-b6de-888e36f0c642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:28.394582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6e9f2f68-51bf-4b8d-b6de-888e36f0c642 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:28.395823 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6e9f2f68-51bf-4b8d-b6de-888e36f0c642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:28.496946 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:28.500513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 666/1348] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:28.538734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302cef8-dddc-40d3-9eaf-fccbf5724041 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:28.540265 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-6302cef8-dddc-40d3-9eaf-fccbf5724041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:28.649919 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302cef8-dddc-40d3-9eaf-fccbf5724041 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:28.699606 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:28.726235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f4a17-9caa-4df3-823e-d116f6ea7af6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:28.726884 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f4a17-9caa-4df3-823e-d116f6ea7af6 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:28.727165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f4a17-9caa-4df3-823e-d116f6ea7af6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:28.727439 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f4a17-9caa-4df3-823e-d116f6ea7af6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:28.729093 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-732f4a17-9caa-4df3-823e-d116f6ea7af6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:28.844651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f4a17-9caa-4df3-823e-d116f6ea7af6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:28.846106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f4a17-9caa-4df3-823e-d116f6ea7af6 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:28.865501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 683/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:22:33.782094 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2d295-1c78-41c3-bd35-21d75519786b None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:33.782593 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2d295-1c78-41c3-bd35-21d75519786b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:33.782593 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2d295-1c78-41c3-bd35-21d75519786b None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:33.784256 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 667/1350] 10.222.0.42 () {54 vars in 921 bytes} [Tue Jul 19 15:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:33.789349 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044a869-2863-4a02-ab94-e1b37153f00b None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:33.790091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044a869-2863-4a02-ab94-e1b37153f00b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:33.790379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044a869-2863-4a02-ab94-e1b37153f00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:33.915222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-9044a869-2863-4a02-ab94-e1b37153f00b None None] MFA Rules not processed for user `6002f32936944b0db6fd04f3851836db`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:33.916324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9044a869-2863-4a02-ab94-e1b37153f00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:33.949760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:34.037441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 684/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Jul 19 15:22:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:38.393519 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177c324-978a-46a1-a6de-f4f9ccb54f44 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:38.394428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177c324-978a-46a1-a6de-f4f9ccb54f44 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:38.394740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177c324-978a-46a1-a6de-f4f9ccb54f44 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:38.396361 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 668/1352] 10.222.0.42 () {54 vars in 921 bytes} [Tue Jul 19 15:22:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:38.401662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5abf66-fe1a-4e27-a5de-a357e226b155 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:38.402542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5abf66-fe1a-4e27-a5de-a357e226b155 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:38.403105 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5abf66-fe1a-4e27-a5de-a357e226b155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:38.498550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-3d5abf66-fe1a-4e27-a5de-a357e226b155 None None] MFA Rules not processed for user `6002f32936944b0db6fd04f3851836db`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:38.502081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3d5abf66-fe1a-4e27-a5de-a357e226b155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:38.596548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:38.646864 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 685/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Jul 19 15:22:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:39.850619 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5c353-3f7d-4ef4-a6aa-d8909966ab54 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:39.851129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5c353-3f7d-4ef4-a6aa-d8909966ab54 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:39.851248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5c353-3f7d-4ef4-a6aa-d8909966ab54 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:39.852574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 669/1354] 10.222.0.42 () {54 vars in 921 bytes} [Tue Jul 19 15:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:39.858355 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b78a79-469a-4ef0-af0c-f7a47c7ee8f1 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:39.858800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b78a79-469a-4ef0-af0c-f7a47c7ee8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:39.858892 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b78a79-469a-4ef0-af0c-f7a47c7ee8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:39.934324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-29b78a79-469a-4ef0-af0c-f7a47c7ee8f1 None None] MFA Rules not processed for user `6002f32936944b0db6fd04f3851836db`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:39.936708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-29b78a79-469a-4ef0-af0c-f7a47c7ee8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:40.046808 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 686/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Jul 19 15:22:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:43.641535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7b22a-7de5-4c2d-8726-69364345a93c None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:43.642067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7b22a-7de5-4c2d-8726-69364345a93c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:43.642067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7b22a-7de5-4c2d-8726-69364345a93c None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:43.643352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 670/1356] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:43.648720 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb3611-27d0-466e-88f5-8f332f57c539 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:43.649182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb3611-27d0-466e-88f5-8f332f57c539 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:43.649311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb3611-27d0-466e-88f5-8f332f57c539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:43.687664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:43.721631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-bfdb3611-27d0-466e-88f5-8f332f57c539 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:43.722834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-bfdb3611-27d0-466e-88f5-8f332f57c539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:43.830689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 687/1357] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:43.884507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd6b52-60bd-45b8-acb1-d70ceab05eed None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:43.884795 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd6b52-60bd-45b8-acb1-d70ceab05eed None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:43.885353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd6b52-60bd-45b8-acb1-d70ceab05eed None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:43.887174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 671/1358] 10.222.0.42 () {54 vars in 962 bytes} [Tue Jul 19 15:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:43.896242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5786134-2d34-469f-acef-88bf1fccdd8c None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:43.896719 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5786134-2d34-469f-acef-88bf1fccdd8c None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:43.897206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5786134-2d34-469f-acef-88bf1fccdd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:43.983570 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-b5786134-2d34-469f-acef-88bf1fccdd8c None None] MFA Rules not processed for user `bff6a2d4bc734904b2615608a86b097d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:43.984778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b5786134-2d34-469f-acef-88bf1fccdd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:44.036729 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:44.112003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 688/1359] 10.222.0.42 () {58 vars in 1059 bytes} [Tue Jul 19 15:22:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:44.120512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd6b52-60bd-45b8-acb1-d70ceab05eed None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:44.122674 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f7dd6b52-60bd-45b8-acb1-d70ceab05eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:44.280916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd6b52-60bd-45b8-acb1-d70ceab05eed None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:44.362217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:44.407096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068e4cb-d72f-4d49-a857-aecf26fa24fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:44.407802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068e4cb-d72f-4d49-a857-aecf26fa24fd service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:44.408085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068e4cb-d72f-4d49-a857-aecf26fa24fd service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:44.408360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068e4cb-d72f-4d49-a857-aecf26fa24fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:44.410210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c068e4cb-d72f-4d49-a857-aecf26fa24fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:44.506199 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068e4cb-d72f-4d49-a857-aecf26fa24fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:44.507720 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068e4cb-d72f-4d49-a857-aecf26fa24fd service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:44.526723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 672/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:22:46.814275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3abb5-5e19-45c0-b5a1-792b456b7802 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:46.814801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3abb5-5e19-45c0-b5a1-792b456b7802 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:46.814801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3abb5-5e19-45c0-b5a1-792b456b7802 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:46.816151 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 689/1361] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:46.821346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7081b-5f39-4b46-bd0e-d87008ddf45a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:46.821757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7081b-5f39-4b46-bd0e-d87008ddf45a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:46.821950 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7081b-5f39-4b46-bd0e-d87008ddf45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:46.895643 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-10f7081b-5f39-4b46-bd0e-d87008ddf45a None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:46.897007 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-10f7081b-5f39-4b46-bd0e-d87008ddf45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:46.938799 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:47.011342 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 673/1362] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:22:47.071318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3abb5-5e19-45c0-b5a1-792b456b7802 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:47.072992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-03c3abb5-5e19-45c0-b5a1-792b456b7802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:47.267558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3abb5-5e19-45c0-b5a1-792b456b7802 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:47.277061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:47.393748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55467e-f1e6-4aeb-8f83-770c6a47da0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:47.394404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55467e-f1e6-4aeb-8f83-770c6a47da0f service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:47.394760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55467e-f1e6-4aeb-8f83-770c6a47da0f service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:47.395106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55467e-f1e6-4aeb-8f83-770c6a47da0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:47.396817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cc55467e-f1e6-4aeb-8f83-770c6a47da0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:47.494142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55467e-f1e6-4aeb-8f83-770c6a47da0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:47.495848 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55467e-f1e6-4aeb-8f83-770c6a47da0f service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:47.516761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 690/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:22:49.907052 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57646d8-1977-44f2-869c-bf60111a08ca None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:49.907589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57646d8-1977-44f2-869c-bf60111a08ca None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:49.907589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57646d8-1977-44f2-869c-bf60111a08ca None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:49.908899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 674/1364] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:49.914328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe169b-03c4-45a0-82e4-53c7f3566c7c None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:49.914749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe169b-03c4-45a0-82e4-53c7f3566c7c None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:49.914931 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe169b-03c4-45a0-82e4-53c7f3566c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:49.930586 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:49.986877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-cdbe169b-03c4-45a0-82e4-53c7f3566c7c None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:49.988085 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cdbe169b-03c4-45a0-82e4-53c7f3566c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:50.093752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 691/1365] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:50.141405 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75413bee-ad7e-45d7-9a22-186cefd31eef None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:50.141847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75413bee-ad7e-45d7-9a22-186cefd31eef None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:50.142647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75413bee-ad7e-45d7-9a22-186cefd31eef None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:50.144123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 675/1366] 10.222.0.42 () {54 vars in 962 bytes} [Tue Jul 19 15:22:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:50.149856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c790e6a-f46a-42b5-a1ba-561f6aace201 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:50.150333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c790e6a-f46a-42b5-a1ba-561f6aace201 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:50.150611 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c790e6a-f46a-42b5-a1ba-561f6aace201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:50.228921 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-3c790e6a-f46a-42b5-a1ba-561f6aace201 None None] MFA Rules not processed for user `bff6a2d4bc734904b2615608a86b097d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:50.230164 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3c790e6a-f46a-42b5-a1ba-561f6aace201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:50.280792 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:50.374558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 692/1367] 10.222.0.42 () {58 vars in 1059 bytes} [Tue Jul 19 15:22:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:50.382934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75413bee-ad7e-45d7-9a22-186cefd31eef None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:50.384657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-75413bee-ad7e-45d7-9a22-186cefd31eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:50.540506 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75413bee-ad7e-45d7-9a22-186cefd31eef None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:50.564319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:50.665351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef59a5-abde-4e89-962b-99889afe0d42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:50.666018 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef59a5-abde-4e89-962b-99889afe0d42 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:50.666295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef59a5-abde-4e89-962b-99889afe0d42 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:50.666572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef59a5-abde-4e89-962b-99889afe0d42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:50.668256 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d4ef59a5-abde-4e89-962b-99889afe0d42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:50.766404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef59a5-abde-4e89-962b-99889afe0d42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:50.768016 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef59a5-abde-4e89-962b-99889afe0d42 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:50.787177 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 676/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:22:51.705174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f9cb9-dc7c-4653-9ef4-9b8f77fa0c6a None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:51.705783 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f9cb9-dc7c-4653-9ef4-9b8f77fa0c6a None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:51.705783 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f9cb9-dc7c-4653-9ef4-9b8f77fa0c6a None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:51.707292 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 693/1369] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:51.712250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3a396-f281-4626-b3dd-397bfb815614 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:51.712675 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3a396-f281-4626-b3dd-397bfb815614 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:51.712848 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3a396-f281-4626-b3dd-397bfb815614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:51.744407 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:51.782217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3bb3a396-f281-4626-b3dd-397bfb815614 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:51.783430 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3bb3a396-f281-4626-b3dd-397bfb815614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:51.885430 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 677/1370] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:52.235331 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec38989-28ca-48ac-b5c5-fad013219b2f None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:52.235621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec38989-28ca-48ac-b5c5-fad013219b2f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:52.235901 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec38989-28ca-48ac-b5c5-fad013219b2f None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:52.238381 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 694/1371] 10.222.0.42 () {54 vars in 964 bytes} [Tue Jul 19 15:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:52.246384 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0dc91-77d8-4ab4-bcb0-715042c55fec None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:52.246790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0dc91-77d8-4ab4-bcb0-715042c55fec None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:52.247111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0dc91-77d8-4ab4-bcb0-715042c55fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:52.330732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-fdd0dc91-77d8-4ab4-bcb0-715042c55fec None None] MFA Rules not processed for user `7245ef27d2454927a01a7cd56a4c27aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:52.331939 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fdd0dc91-77d8-4ab4-bcb0-715042c55fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:52.381356 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:52.415213 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 678/1372] 10.222.0.42 () {58 vars in 1061 bytes} [Tue Jul 19 15:22:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:52.422778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec38989-28ca-48ac-b5c5-fad013219b2f None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:52.424278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cec38989-28ca-48ac-b5c5-fad013219b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:52.540713 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec38989-28ca-48ac-b5c5-fad013219b2f None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:52.585476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:52.631447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b503d70-96c9-4204-841b-a2a59a71e62b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:52.632212 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b503d70-96c9-4204-841b-a2a59a71e62b service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:52.632502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b503d70-96c9-4204-841b-a2a59a71e62b service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:52.632782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b503d70-96c9-4204-841b-a2a59a71e62b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:52.634453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2b503d70-96c9-4204-841b-a2a59a71e62b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:52.776352 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b503d70-96c9-4204-841b-a2a59a71e62b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:52.778209 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b503d70-96c9-4204-841b-a2a59a71e62b service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:52.800241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 695/1373] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:22:52.975358 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a91a06-5964-4ac0-862f-72ce8f29f5ac None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:52.975818 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a91a06-5964-4ac0-862f-72ce8f29f5ac None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:52.975933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a91a06-5964-4ac0-862f-72ce8f29f5ac None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:52.977333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 679/1374] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:52.983466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb29b8-d646-4ab8-84ef-518fdc699ec4 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:52.983793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb29b8-d646-4ab8-84ef-518fdc699ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:52.984152 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb29b8-d646-4ab8-84ef-518fdc699ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:53.056814 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-6bdb29b8-d646-4ab8-84ef-518fdc699ec4 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:53.057987 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-6bdb29b8-d646-4ab8-84ef-518fdc699ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:53.080365 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:53.169184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 696/1375] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:53.208417 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a91a06-5964-4ac0-862f-72ce8f29f5ac None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:53.209965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-25a91a06-5964-4ac0-862f-72ce8f29f5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:53.358788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a91a06-5964-4ac0-862f-72ce8f29f5ac None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:53.410527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:53.472812 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f51be-9ee5-4241-87b2-ed98597f5d6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:53.473499 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f51be-9ee5-4241-87b2-ed98597f5d6c service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:53.473787 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f51be-9ee5-4241-87b2-ed98597f5d6c service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:53.474066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f51be-9ee5-4241-87b2-ed98597f5d6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:53.475731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b42f51be-9ee5-4241-87b2-ed98597f5d6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:53.568216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f51be-9ee5-4241-87b2-ed98597f5d6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:53.569852 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f51be-9ee5-4241-87b2-ed98597f5d6c service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:53.588654 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 680/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:22:55.660140 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb1120-2e92-4527-8224-c64dfe5dfe34 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:55.660635 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb1120-2e92-4527-8224-c64dfe5dfe34 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:55.660635 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb1120-2e92-4527-8224-c64dfe5dfe34 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:55.662224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 697/1377] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:55.667767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588d8b8-baea-42aa-a837-69c9fb291f8d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:55.668209 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588d8b8-baea-42aa-a837-69c9fb291f8d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:55.668308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588d8b8-baea-42aa-a837-69c9fb291f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:55.742435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6588d8b8-baea-42aa-a837-69c9fb291f8d None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:55.743708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6588d8b8-baea-42aa-a837-69c9fb291f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:55.751532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:55.849040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 681/1378] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:55.901552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb1120-2e92-4527-8224-c64dfe5dfe34 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:55.903798 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-8ceb1120-2e92-4527-8224-c64dfe5dfe34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:56.053577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb1120-2e92-4527-8224-c64dfe5dfe34 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:56.057184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:56.168314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefda05c-0170-4396-9955-97f10c738f2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:56.168965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefda05c-0170-4396-9955-97f10c738f2e service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:56.169276 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefda05c-0170-4396-9955-97f10c738f2e service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:56.169553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefda05c-0170-4396-9955-97f10c738f2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:56.171176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cefda05c-0170-4396-9955-97f10c738f2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:56.282419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefda05c-0170-4396-9955-97f10c738f2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:56.284038 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefda05c-0170-4396-9955-97f10c738f2e service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:56.302038 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 698/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:22:58.441428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afe22e-a068-4ceb-be76-a1d1852ff96a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:58.441972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afe22e-a068-4ceb-be76-a1d1852ff96a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:58.441972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afe22e-a068-4ceb-be76-a1d1852ff96a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:58.443604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 682/1380] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:22:58.449413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7bb31-e7a0-4852-92c9-fc2fa9dac827 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:58.449955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7bb31-e7a0-4852-92c9-fc2fa9dac827 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:58.450201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7bb31-e7a0-4852-92c9-fc2fa9dac827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:58.460979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:58.521783 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-27d7bb31-e7a0-4852-92c9-fc2fa9dac827 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:22:58.523028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-27d7bb31-e7a0-4852-92c9-fc2fa9dac827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:58.629797 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 699/1381] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:22:58.674348 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afe22e-a068-4ceb-be76-a1d1852ff96a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:22:58.676006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-32afe22e-a068-4ceb-be76-a1d1852ff96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:58.828824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:58.842416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afe22e-a068-4ceb-be76-a1d1852ff96a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:22:58.969956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bb0dd-d327-4ad6-86d9-17607b2bf206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:22:58.970801 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bb0dd-d327-4ad6-86d9-17607b2bf206 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:22:58.971186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bb0dd-d327-4ad6-86d9-17607b2bf206 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:22:58.971695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bb0dd-d327-4ad6-86d9-17607b2bf206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:22:58.974258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-df6bb0dd-d327-4ad6-86d9-17607b2bf206 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:22:59.076666 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bb0dd-d327-4ad6-86d9-17607b2bf206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:22:59.078398 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bb0dd-d327-4ad6-86d9-17607b2bf206 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:22:59.089537 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:22:59.100062 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 683/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:22:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:01.256721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f238bbe-1cfa-4672-82ea-3c64fd89843d None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:01.257245 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f238bbe-1cfa-4672-82ea-3c64fd89843d None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:01.257616 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f238bbe-1cfa-4672-82ea-3c64fd89843d None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:01.259446 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 700/1383] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:23:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:01.265335 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099c584-1b21-44cd-aa4f-810a9c8c33e2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:01.265792 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099c584-1b21-44cd-aa4f-810a9c8c33e2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:01.266056 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099c584-1b21-44cd-aa4f-810a9c8c33e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:01.334615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b099c584-1b21-44cd-aa4f-810a9c8c33e2 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:01.335818 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b099c584-1b21-44cd-aa4f-810a9c8c33e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:01.437122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 684/1384] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:01.480041 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f238bbe-1cfa-4672-82ea-3c64fd89843d None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:01.482469 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5f238bbe-1cfa-4672-82ea-3c64fd89843d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:01.568969 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:01.636847 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f238bbe-1cfa-4672-82ea-3c64fd89843d None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:01.765293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e595e4-9806-45ff-9220-65c12793de09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:01.765971 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e595e4-9806-45ff-9220-65c12793de09 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:01.766258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e595e4-9806-45ff-9220-65c12793de09 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:01.766542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e595e4-9806-45ff-9220-65c12793de09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:01.768197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-30e595e4-9806-45ff-9220-65c12793de09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:01.820160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:01.864177 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e595e4-9806-45ff-9220-65c12793de09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:01.865947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e595e4-9806-45ff-9220-65c12793de09 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:01.884355 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 701/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:03.947083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa76ac-daea-4bf4-b414-facbe2eb5f51 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:03.947623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa76ac-daea-4bf4-b414-facbe2eb5f51 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:03.947623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa76ac-daea-4bf4-b414-facbe2eb5f51 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:03.948944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 685/1386] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:03.954452 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fd3f0-f864-4f1d-94eb-a5999bfe1dbd None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:03.954886 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fd3f0-f864-4f1d-94eb-a5999bfe1dbd None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:03.955118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fd3f0-f864-4f1d-94eb-a5999bfe1dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:04.025118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-7c1fd3f0-f864-4f1d-94eb-a5999bfe1dbd None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:04.026388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7c1fd3f0-f864-4f1d-94eb-a5999bfe1dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:04.126841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 702/1387] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:23:04.173896 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa76ac-daea-4bf4-b414-facbe2eb5f51 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:04.175521 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d6fa76ac-daea-4bf4-b414-facbe2eb5f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:04.262360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:04.345096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa76ac-daea-4bf4-b414-facbe2eb5f51 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:04.464301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9161ec9-a438-4a56-90ff-d3abd3bd26bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:04.465612 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9161ec9-a438-4a56-90ff-d3abd3bd26bd service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:04.465898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9161ec9-a438-4a56-90ff-d3abd3bd26bd service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:04.466178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9161ec9-a438-4a56-90ff-d3abd3bd26bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:04.467866 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a9161ec9-a438-4a56-90ff-d3abd3bd26bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:04.507005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:04.561999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9161ec9-a438-4a56-90ff-d3abd3bd26bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:04.563676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9161ec9-a438-4a56-90ff-d3abd3bd26bd service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:04.582185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 686/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:06.665743 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f51bd0-9945-481a-aa03-54c6b20e36a5 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:06.666374 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f51bd0-9945-481a-aa03-54c6b20e36a5 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:06.666374 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f51bd0-9945-481a-aa03-54c6b20e36a5 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:06.667831 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 703/1389] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:06.672883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c715752-c589-4db4-b0e8-6d08d147164d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:06.673314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c715752-c589-4db4-b0e8-6d08d147164d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:06.673478 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c715752-c589-4db4-b0e8-6d08d147164d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:06.740676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8c715752-c589-4db4-b0e8-6d08d147164d None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:06.741920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8c715752-c589-4db4-b0e8-6d08d147164d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:06.841345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 687/1390] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:06.886246 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f51bd0-9945-481a-aa03-54c6b20e36a5 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:06.888033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-67f51bd0-9945-481a-aa03-54c6b20e36a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:06.919689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:07.038338 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f51bd0-9945-481a-aa03-54c6b20e36a5 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:07.146721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:07.159684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf64f23-486e-4657-87a1-c8e91f0f3c25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:07.160336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf64f23-486e-4657-87a1-c8e91f0f3c25 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:07.160620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf64f23-486e-4657-87a1-c8e91f0f3c25 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:07.160897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf64f23-486e-4657-87a1-c8e91f0f3c25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:07.162575 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0bf64f23-486e-4657-87a1-c8e91f0f3c25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:07.254244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf64f23-486e-4657-87a1-c8e91f0f3c25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:07.255879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf64f23-486e-4657-87a1-c8e91f0f3c25 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:07.275086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 704/1391] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:08.036481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5642b-7e13-4a84-9f1e-8b184e9eb401 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:08.037042 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5642b-7e13-4a84-9f1e-8b184e9eb401 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:08.037042 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5642b-7e13-4a84-9f1e-8b184e9eb401 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:08.038510 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 688/1392] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:08.043822 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a25fa-10c5-4aa0-a2ca-c3382afda734 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:08.044275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a25fa-10c5-4aa0-a2ca-c3382afda734 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:08.044519 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a25fa-10c5-4aa0-a2ca-c3382afda734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:08.109905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-026a25fa-10c5-4aa0-a2ca-c3382afda734 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:08.111098 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-026a25fa-10c5-4aa0-a2ca-c3382afda734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:08.170440 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:08.210503 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 705/1393] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:23:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:08.268829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738dc9b1-8d18-4c3c-aa1c-55570721a239 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:08.269098 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738dc9b1-8d18-4c3c-aa1c-55570721a239 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:08.269404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738dc9b1-8d18-4c3c-aa1c-55570721a239 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:08.270785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 689/1394] 10.222.0.42 () {54 vars in 961 bytes} [Tue Jul 19 15:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:08.275936 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973ae5e-2e1a-4350-a4c1-7ffea5081214 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:08.276316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973ae5e-2e1a-4350-a4c1-7ffea5081214 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:08.276696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973ae5e-2e1a-4350-a4c1-7ffea5081214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:08.367453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-1973ae5e-2e1a-4350-a4c1-7ffea5081214 None None] MFA Rules not processed for user `215c6f6f1aad486f85ce20590d889cbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:08.369102 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1973ae5e-2e1a-4350-a4c1-7ffea5081214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:08.457803 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 706/1395] 10.222.0.42 () {58 vars in 1058 bytes} [Tue Jul 19 15:23:08 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:23:08.466587 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738dc9b1-8d18-4c3c-aa1c-55570721a239 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:08.468132 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-738dc9b1-8d18-4c3c-aa1c-55570721a239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:08.483005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:08.575263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738dc9b1-8d18-4c3c-aa1c-55570721a239 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:08.649712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6510ca0-f6c9-4516-ab4d-78619804e34c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '215c6f6f1aad486f85ce20590d889cbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:08.650296 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6510ca0-f6c9-4516-ab4d-78619804e34c service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:08.650585 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6510ca0-f6c9-4516-ab4d-78619804e34c service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:08.650868 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6510ca0-f6c9-4516-ab4d-78619804e34c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:08.653275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b6510ca0-f6c9-4516-ab4d-78619804e34c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:08.725988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:08.770062 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6510ca0-f6c9-4516-ab4d-78619804e34c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:08.771512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6510ca0-f6c9-4516-ab4d-78619804e34c service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:08.790676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 690/1396] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:23:09.344994 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4911d7-0332-403e-8bf3-3327cb26b0b6 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:09.345609 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4911d7-0332-403e-8bf3-3327cb26b0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:09.345609 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4911d7-0332-403e-8bf3-3327cb26b0b6 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:09.346996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 707/1397] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:09.352871 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1f616-e2c9-4d6b-9fa5-639c4aa184f0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:09.353437 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1f616-e2c9-4d6b-9fa5-639c4aa184f0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:09.353551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1f616-e2c9-4d6b-9fa5-639c4aa184f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:09.428867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-99d1f616-e2c9-4d6b-9fa5-639c4aa184f0 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:09.430246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-99d1f616-e2c9-4d6b-9fa5-639c4aa184f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:09.509235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16c6f5-eccf-40b4-951d-2a193941cf08 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:09.509235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16c6f5-eccf-40b4-951d-2a193941cf08 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:09.510011 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16c6f5-eccf-40b4-951d-2a193941cf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:09.532744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 691/1398] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:09.578099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1f616-e2c9-4d6b-9fa5-639c4aa184f0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:09.579781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-99d1f616-e2c9-4d6b-9fa5-639c4aa184f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:09.581404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-fb16c6f5-eccf-40b4-951d-2a193941cf08 None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:09.582577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-fb16c6f5-eccf-40b4-951d-2a193941cf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:09.590922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:09.607831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:09.671478 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 708/1399] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:23:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:23:09.699908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1f616-e2c9-4d6b-9fa5-639c4aa184f0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:09.714865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fb248-110a-478e-a755-05272f1dbc28 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:09.715886 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fb248-110a-478e-a755-05272f1dbc28 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:09.716091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fb248-110a-478e-a755-05272f1dbc28 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:09.717517 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 709/1400] 10.222.0.42 () {54 vars in 943 bytes} [Tue Jul 19 15:23:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:23:09.723297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a51e30-7984-4eaf-80b6-aa510447e819 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:09.723584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a51e30-7984-4eaf-80b6-aa510447e819 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:09.723863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a51e30-7984-4eaf-80b6-aa510447e819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:09.801829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-33a51e30-7984-4eaf-80b6-aa510447e819 None None] MFA Rules not processed for user `96faaac55bc5408685c68a6177d91971`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:09.802944 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-33a51e30-7984-4eaf-80b6-aa510447e819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:09.817638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51112bbf-1fcf-4c46-83be-663dc1a71e32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:09.818295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51112bbf-1fcf-4c46-83be-663dc1a71e32 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:09.818583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51112bbf-1fcf-4c46-83be-663dc1a71e32 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:09.818864 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51112bbf-1fcf-4c46-83be-663dc1a71e32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:09.820510 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-51112bbf-1fcf-4c46-83be-663dc1a71e32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:09.838568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:09.891005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:09.913647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51112bbf-1fcf-4c46-83be-663dc1a71e32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:09.914171 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 710/1401] 10.222.0.42 () {58 vars in 1040 bytes} [Tue Jul 19 15:23:09 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:23:09.915268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51112bbf-1fcf-4c46-83be-663dc1a71e32 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:09.923985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a51e30-7984-4eaf-80b6-aa510447e819 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:09.933534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a51e30-7984-4eaf-80b6-aa510447e819 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:09.935139 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 692/1402] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:23:10.054596 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014e1f6-13e0-41ba-b970-bea8e70e6c2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:10.055591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014e1f6-13e0-41ba-b970-bea8e70e6c2d service swift] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:10.056053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014e1f6-13e0-41ba-b970-bea8e70e6c2d service swift] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:10.056488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014e1f6-13e0-41ba-b970-bea8e70e6c2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:10.058545 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7014e1f6-13e0-41ba-b970-bea8e70e6c2d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:10.150574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014e1f6-13e0-41ba-b970-bea8e70e6c2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:10.152186 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014e1f6-13e0-41ba-b970-bea8e70e6c2d service swift] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:10.161132 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 711/1403] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:23:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 19 15:23:12.075969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69142047-223f-4cdc-b4ff-4ebc7296f574 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:12.076596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69142047-223f-4cdc-b4ff-4ebc7296f574 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:12.076596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69142047-223f-4cdc-b4ff-4ebc7296f574 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:12.077961 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 693/1404] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:12.083235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd058d0f-0bb0-4816-8b5b-e74ca0fb03c4 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:12.083682 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd058d0f-0bb0-4816-8b5b-e74ca0fb03c4 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:12.083813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd058d0f-0bb0-4816-8b5b-e74ca0fb03c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:12.094868 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:12.155048 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-dd058d0f-0bb0-4816-8b5b-e74ca0fb03c4 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:12.156305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-dd058d0f-0bb0-4816-8b5b-e74ca0fb03c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:12.278672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 712/1405] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:12.318065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69142047-223f-4cdc-b4ff-4ebc7296f574 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:12.319638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-69142047-223f-4cdc-b4ff-4ebc7296f574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:12.488971 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:12.493944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69142047-223f-4cdc-b4ff-4ebc7296f574 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:12.622049 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedfd78-348a-4889-9769-6dd910bf0dcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:12.623155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedfd78-348a-4889-9769-6dd910bf0dcf service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:12.623562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedfd78-348a-4889-9769-6dd910bf0dcf service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:12.623850 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedfd78-348a-4889-9769-6dd910bf0dcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:12.625529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9cedfd78-348a-4889-9769-6dd910bf0dcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:12.734824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedfd78-348a-4889-9769-6dd910bf0dcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:12.736577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedfd78-348a-4889-9769-6dd910bf0dcf service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:12.762151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 694/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:14.857875 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ccc49-6b0e-496e-8116-5b186756b9e1 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:14.858359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ccc49-6b0e-496e-8116-5b186756b9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:14.858450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ccc49-6b0e-496e-8116-5b186756b9e1 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:14.861320 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 713/1407] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:23:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:14.865045 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82608111-60dd-46c4-913e-89ff84d73432 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:14.865572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82608111-60dd-46c4-913e-89ff84d73432 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:14.865661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82608111-60dd-46c4-913e-89ff84d73432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:14.868858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:14.935463 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-82608111-60dd-46c4-913e-89ff84d73432 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:14.936712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-82608111-60dd-46c4-913e-89ff84d73432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:15.045649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 695/1408] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:15.095311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ccc49-6b0e-496e-8116-5b186756b9e1 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:15.099000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-fe5ccc49-6b0e-496e-8116-5b186756b9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:15.218600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:15.305692 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ccc49-6b0e-496e-8116-5b186756b9e1 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:15.472767 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674a7ca-2ecf-4079-a1bd-716b511a5050 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:15.473613 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674a7ca-2ecf-4079-a1bd-716b511a5050 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:15.473959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674a7ca-2ecf-4079-a1bd-716b511a5050 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:15.474303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674a7ca-2ecf-4079-a1bd-716b511a5050 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:15.476016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f674a7ca-2ecf-4079-a1bd-716b511a5050 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:15.529674 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:15.573702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674a7ca-2ecf-4079-a1bd-716b511a5050 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:15.576220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674a7ca-2ecf-4079-a1bd-716b511a5050 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:15.599937 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 714/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:25.043210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=64 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:25.594040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=23 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:32.520835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80440d-1429-4bf3-95d6-d24dfcbd4dcb None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:32.521376 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80440d-1429-4bf3-95d6-d24dfcbd4dcb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:32.521492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80440d-1429-4bf3-95d6-d24dfcbd4dcb None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:32.522891 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 696/1410] 10.222.0.42 () {54 vars in 918 bytes} [Tue Jul 19 15:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:32.528142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0504fcc-d051-4099-b296-32ac1c25137b None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:32.529086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0504fcc-d051-4099-b296-32ac1c25137b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:32.529385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0504fcc-d051-4099-b296-32ac1c25137b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:32.603238 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-b0504fcc-d051-4099-b296-32ac1c25137b None None] MFA Rules not processed for user `6002f32936944b0db6fd04f3851836db`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:32.604445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b0504fcc-d051-4099-b296-32ac1c25137b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:32.710470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 715/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Tue Jul 19 15:23:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:32.725860 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a36e0f-04f7-4e4d-acb9-2f3063ebeb59 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:32.726146 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a36e0f-04f7-4e4d-acb9-2f3063ebeb59 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:32.726426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a36e0f-04f7-4e4d-acb9-2f3063ebeb59 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:32.727789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 697/1412] 10.222.0.42 () {54 vars in 918 bytes} [Tue Jul 19 15:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:32.732673 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ba9e5-b266-4fd8-b321-4e31a8e0ddff None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:32.732959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ba9e5-b266-4fd8-b321-4e31a8e0ddff None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:32.733263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ba9e5-b266-4fd8-b321-4e31a8e0ddff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:32.803916 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:32.806432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-b95ba9e5-b266-4fd8-b321-4e31a8e0ddff None None] MFA Rules not processed for user `bff6a2d4bc734904b2615608a86b097d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:32.808153 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b95ba9e5-b266-4fd8-b321-4e31a8e0ddff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:32.924075 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 716/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Tue Jul 19 15:23:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:23:35.411035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac035504-64eb-4507-a8d0-fa59a90d0923 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:35.411731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac035504-64eb-4507-a8d0-fa59a90d0923 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:35.411731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac035504-64eb-4507-a8d0-fa59a90d0923 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:35.412949 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 15:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:35.418644 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9df3af-27a7-4fba-80ac-24b8227d11b4 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:35.419065 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9df3af-27a7-4fba-80ac-24b8227d11b4 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:35.419239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9df3af-27a7-4fba-80ac-24b8227d11b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:35.483130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-4f9df3af-27a7-4fba-80ac-24b8227d11b4 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:35.484323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-4f9df3af-27a7-4fba-80ac-24b8227d11b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:35.539372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:35.581556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 15:23:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:35.628574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac035504-64eb-4507-a8d0-fa59a90d0923 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:35.630296 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ac035504-64eb-4507-a8d0-fa59a90d0923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:35.778212 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac035504-64eb-4507-a8d0-fa59a90d0923 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:35.849218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:35.889155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e988e0d-eabf-4d0a-bb93-9bf67bf2742b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:35.889797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e988e0d-eabf-4d0a-bb93-9bf67bf2742b service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:35.890080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e988e0d-eabf-4d0a-bb93-9bf67bf2742b service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:35.890357 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e988e0d-eabf-4d0a-bb93-9bf67bf2742b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:35.892259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1e988e0d-eabf-4d0a-bb93-9bf67bf2742b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:35.981285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e988e0d-eabf-4d0a-bb93-9bf67bf2742b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:35.983482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e988e0d-eabf-4d0a-bb93-9bf67bf2742b service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:36.001229 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 699/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:36.918548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce84e7-dab4-47ee-9520-fb2d02b57e5e None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:36.919051 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce84e7-dab4-47ee-9520-fb2d02b57e5e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:36.919157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce84e7-dab4-47ee-9520-fb2d02b57e5e None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:36.920661 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 15:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:36.925667 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08305ed-d565-432f-9bbf-5c326502342f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:36.926075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08305ed-d565-432f-9bbf-5c326502342f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:36.926262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08305ed-d565-432f-9bbf-5c326502342f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:37.001350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b08305ed-d565-432f-9bbf-5c326502342f None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:37.002566 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b08305ed-d565-432f-9bbf-5c326502342f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:37.029814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:37.100251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 15:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:37.251887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce84e7-dab4-47ee-9520-fb2d02b57e5e None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:37.253448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e3ce84e7-dab4-47ee-9520-fb2d02b57e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:37.402058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce84e7-dab4-47ee-9520-fb2d02b57e5e None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:37.442810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:37.515919 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c039831-c34a-4e19-a607-abe4db6e8eda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:37.516747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c039831-c34a-4e19-a607-abe4db6e8eda service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:37.517025 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c039831-c34a-4e19-a607-abe4db6e8eda service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:37.517342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c039831-c34a-4e19-a607-abe4db6e8eda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:37.518982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5c039831-c34a-4e19-a607-abe4db6e8eda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:37.606665 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c039831-c34a-4e19-a607-abe4db6e8eda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:37.608734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c039831-c34a-4e19-a607-abe4db6e8eda service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:37.627417 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 719/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:38.522158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673e428-17c6-4ae7-9816-66f2b1aab787 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:38.522691 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673e428-17c6-4ae7-9816-66f2b1aab787 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:38.522691 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673e428-17c6-4ae7-9816-66f2b1aab787 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:38.524059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 15:23:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:38.530415 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7d881-e763-47d8-96b6-d45414f11fac None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:38.530850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7d881-e763-47d8-96b6-d45414f11fac None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:38.531058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7d881-e763-47d8-96b6-d45414f11fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:38.587163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:38.610847 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ccd7d881-e763-47d8-96b6-d45414f11fac None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:38.612108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ccd7d881-e763-47d8-96b6-d45414f11fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:38.714352 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 15:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:38.867116 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673e428-17c6-4ae7-9816-66f2b1aab787 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:38.868653 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b673e428-17c6-4ae7-9816-66f2b1aab787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:39.019673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673e428-17c6-4ae7-9816-66f2b1aab787 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:39.032495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:39.140391 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9909a3cc-3935-44fa-8d32-f2dc9bf2d6eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:39.141056 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909a3cc-3935-44fa-8d32-f2dc9bf2d6eb service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:39.141627 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909a3cc-3935-44fa-8d32-f2dc9bf2d6eb service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:39.141900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909a3cc-3935-44fa-8d32-f2dc9bf2d6eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:39.143546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9909a3cc-3935-44fa-8d32-f2dc9bf2d6eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:39.236983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9909a3cc-3935-44fa-8d32-f2dc9bf2d6eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:39.238634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9909a3cc-3935-44fa-8d32-f2dc9bf2d6eb service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:39.256784 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 702/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:40.118993 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282df6e5-a0f7-485f-9ea0-cfa5d24bb37c None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:40.119530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282df6e5-a0f7-485f-9ea0-cfa5d24bb37c None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:40.119530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282df6e5-a0f7-485f-9ea0-cfa5d24bb37c None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:40.120916 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 15:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:40.126048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4cdfac-2c90-4dbf-8320-64266a9448a7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:40.126459 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4cdfac-2c90-4dbf-8320-64266a9448a7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:40.126608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4cdfac-2c90-4dbf-8320-64266a9448a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:40.146445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:40.203419 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0a4cdfac-2c90-4dbf-8320-64266a9448a7 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:40.204729 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0a4cdfac-2c90-4dbf-8320-64266a9448a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:40.302720 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 15:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:40.453688 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282df6e5-a0f7-485f-9ea0-cfa5d24bb37c None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:40.455191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-282df6e5-a0f7-485f-9ea0-cfa5d24bb37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:40.565694 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:40.604731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282df6e5-a0f7-485f-9ea0-cfa5d24bb37c None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:40.716501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218b90a-5d7d-404c-bca6-18fabcb8b5b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:40.717081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218b90a-5d7d-404c-bca6-18fabcb8b5b1 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:40.717456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218b90a-5d7d-404c-bca6-18fabcb8b5b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:40.717732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218b90a-5d7d-404c-bca6-18fabcb8b5b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:40.719398 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e218b90a-5d7d-404c-bca6-18fabcb8b5b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:40.794958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:40.808678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218b90a-5d7d-404c-bca6-18fabcb8b5b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:40.810952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218b90a-5d7d-404c-bca6-18fabcb8b5b1 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:40.828572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 722/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:41.685322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e6e94-53dd-4e2f-afd8-c02bb1398f8b None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:41.686028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e6e94-53dd-4e2f-afd8-c02bb1398f8b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:41.686156 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e6e94-53dd-4e2f-afd8-c02bb1398f8b None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:41.687613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 15:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:41.692532 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddeb71-4e73-4cff-9331-7b18970e1fea None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:41.692976 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddeb71-4e73-4cff-9331-7b18970e1fea None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:41.693072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddeb71-4e73-4cff-9331-7b18970e1fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:41.767497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-e9ddeb71-4e73-4cff-9331-7b18970e1fea None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:41.768693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e9ddeb71-4e73-4cff-9331-7b18970e1fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:41.862637 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 15:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:42.015137 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e6e94-53dd-4e2f-afd8-c02bb1398f8b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:42.017000 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f26e6e94-53dd-4e2f-afd8-c02bb1398f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:42.102693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:42.166300 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e6e94-53dd-4e2f-afd8-c02bb1398f8b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:42.277539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2154f-818d-4643-883e-71dd01907024 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:42.278151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2154f-818d-4643-883e-71dd01907024 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:42.278434 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2154f-818d-4643-883e-71dd01907024 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:42.278710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2154f-818d-4643-883e-71dd01907024 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:42.281096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b3e2154f-818d-4643-883e-71dd01907024 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:42.329916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:42.371790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2154f-818d-4643-883e-71dd01907024 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:42.373597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2154f-818d-4643-883e-71dd01907024 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:42.392430 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 705/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:23:43.199147 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3ee3c-e777-487d-902d-7c28295788bc None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:43.199685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3ee3c-e777-487d-902d-7c28295788bc None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:43.199685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3ee3c-e777-487d-902d-7c28295788bc None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:43.201543 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 15:23:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:43.206545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de790aab-65d8-4e0e-8c41-1294c87404fe None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:43.206948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de790aab-65d8-4e0e-8c41-1294c87404fe None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:43.207102 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de790aab-65d8-4e0e-8c41-1294c87404fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:43.282053 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-de790aab-65d8-4e0e-8c41-1294c87404fe None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:43.283253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-de790aab-65d8-4e0e-8c41-1294c87404fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:43.380663 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 15:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:43.532532 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3ee3c-e777-487d-902d-7c28295788bc None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:43.534083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2fe3ee3c-e777-487d-902d-7c28295788bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:43.581471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:43.679162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3ee3c-e777-487d-902d-7c28295788bc None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:43.911880 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13e55a-02c6-49de-bbb3-d6396dc68e7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:43.912443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13e55a-02c6-49de-bbb3-d6396dc68e7e service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:43.912725 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13e55a-02c6-49de-bbb3-d6396dc68e7e service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:43.913004 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13e55a-02c6-49de-bbb3-d6396dc68e7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:43.914681 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3b13e55a-02c6-49de-bbb3-d6396dc68e7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:43.921717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:44.002734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13e55a-02c6-49de-bbb3-d6396dc68e7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:44.004332 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13e55a-02c6-49de-bbb3-d6396dc68e7e service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:44.022962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 725/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:44.835000 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d5143-7c16-4367-a758-9416152a4342 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:44.835571 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d5143-7c16-4367-a758-9416152a4342 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:44.835571 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d5143-7c16-4367-a758-9416152a4342 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:44.836953 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 15:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:44.842048 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cb91f-e029-46c4-98c5-03c173c82883 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:44.842510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cb91f-e029-46c4-98c5-03c173c82883 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:44.842510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cb91f-e029-46c4-98c5-03c173c82883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:44.917722 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-932cb91f-e029-46c4-98c5-03c173c82883 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:44.918922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-932cb91f-e029-46c4-98c5-03c173c82883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:44.987524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:45.016212 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 15:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:45.169364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d5143-7c16-4367-a758-9416152a4342 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:45.171529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-350d5143-7c16-4367-a758-9416152a4342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:45.189031 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:45.319995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d5143-7c16-4367-a758-9416152a4342 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:45.408376 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:45.435785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca017b-24ff-4039-966b-1752a9839601 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:45.436456 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca017b-24ff-4039-966b-1752a9839601 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:45.436740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca017b-24ff-4039-966b-1752a9839601 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:45.437020 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca017b-24ff-4039-966b-1752a9839601 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:45.438694 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4bca017b-24ff-4039-966b-1752a9839601 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:45.526805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca017b-24ff-4039-966b-1752a9839601 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:45.528412 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca017b-24ff-4039-966b-1752a9839601 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:45.547243 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 708/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:46.400554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2c635-f92e-4734-ac55-3af4c60239a0 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:46.401191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2c635-f92e-4734-ac55-3af4c60239a0 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:46.401191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2c635-f92e-4734-ac55-3af4c60239a0 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:46.402727 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 15:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:46.408704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a85a2-e4ea-40a3-ba5e-a165a37ce835 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:46.409130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a85a2-e4ea-40a3-ba5e-a165a37ce835 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:46.409294 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a85a2-e4ea-40a3-ba5e-a165a37ce835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:46.485403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0c1a85a2-e4ea-40a3-ba5e-a165a37ce835 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:46.486606 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0c1a85a2-e4ea-40a3-ba5e-a165a37ce835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:46.527789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:46.583033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 15:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:46.732607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2c635-f92e-4734-ac55-3af4c60239a0 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:46.734032 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c3e2c635-f92e-4734-ac55-3af4c60239a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:46.878671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2c635-f92e-4734-ac55-3af4c60239a0 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:46.930892 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:46.990704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c7667-228d-43e5-bc3b-c0e628c06a8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:46.991375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c7667-228d-43e5-bc3b-c0e628c06a8d service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:46.991652 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c7667-228d-43e5-bc3b-c0e628c06a8d service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:46.991922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c7667-228d-43e5-bc3b-c0e628c06a8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:46.993577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a22c7667-228d-43e5-bc3b-c0e628c06a8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:47.082961 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c7667-228d-43e5-bc3b-c0e628c06a8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:47.084559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c7667-228d-43e5-bc3b-c0e628c06a8d service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:47.102279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 728/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:47.903290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4a76e-396a-41b3-8f80-7dbb4375c395 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:47.903811 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4a76e-396a-41b3-8f80-7dbb4375c395 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:47.903811 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4a76e-396a-41b3-8f80-7dbb4375c395 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:47.905764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 15:23:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:47.910621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3946004-6a65-4b46-ab9b-e0dbd7f150f5 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:47.911026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3946004-6a65-4b46-ab9b-e0dbd7f150f5 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:47.911169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3946004-6a65-4b46-ab9b-e0dbd7f150f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:47.984865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-e3946004-6a65-4b46-ab9b-e0dbd7f150f5 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:47.986100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e3946004-6a65-4b46-ab9b-e0dbd7f150f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:47.988108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:48.082733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 15:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:48.234234 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4a76e-396a-41b3-8f80-7dbb4375c395 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:48.235766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a5e4a76e-396a-41b3-8f80-7dbb4375c395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:48.381216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4a76e-396a-41b3-8f80-7dbb4375c395 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:48.408708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:48.493586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75fc4d-e76f-4283-be7b-ceaf77ecfe03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:48.494240 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75fc4d-e76f-4283-be7b-ceaf77ecfe03 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:48.494524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75fc4d-e76f-4283-be7b-ceaf77ecfe03 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:48.494802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75fc4d-e76f-4283-be7b-ceaf77ecfe03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:48.496460 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-df75fc4d-e76f-4283-be7b-ceaf77ecfe03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:48.584872 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75fc4d-e76f-4283-be7b-ceaf77ecfe03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:48.586491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75fc4d-e76f-4283-be7b-ceaf77ecfe03 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:48.604611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 711/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:50.533813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d77822-6cac-40bf-aa38-81be2ce141c5 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:50.534360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d77822-6cac-40bf-aa38-81be2ce141c5 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:50.534360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d77822-6cac-40bf-aa38-81be2ce141c5 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:50.535656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 15:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:50.541031 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb756735-c406-46a1-b292-3e68d78447b0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:50.541478 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb756735-c406-46a1-b292-3e68d78447b0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:50.541641 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb756735-c406-46a1-b292-3e68d78447b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:50.581857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:50.607391 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bb756735-c406-46a1-b292-3e68d78447b0 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:50.608602 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bb756735-c406-46a1-b292-3e68d78447b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:50.704233 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 15:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:50.743156 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d77822-6cac-40bf-aa38-81be2ce141c5 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:50.744672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-68d77822-6cac-40bf-aa38-81be2ce141c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:50.846373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d77822-6cac-40bf-aa38-81be2ce141c5 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:50.879577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:50.922210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b68b61-9a3f-46fe-a0d3-48e5b9aaeb3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:50.922854 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b68b61-9a3f-46fe-a0d3-48e5b9aaeb3d service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:50.923138 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b68b61-9a3f-46fe-a0d3-48e5b9aaeb3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:50.923414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b68b61-9a3f-46fe-a0d3-48e5b9aaeb3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:50.925046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b7b68b61-9a3f-46fe-a0d3-48e5b9aaeb3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:51.035343 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b68b61-9a3f-46fe-a0d3-48e5b9aaeb3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:51.036778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b68b61-9a3f-46fe-a0d3-48e5b9aaeb3d service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:51.055323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 731/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:53.280607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b5d89-3512-4beb-a919-5105e655d41f None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:53.281093 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b5d89-3512-4beb-a919-5105e655d41f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:53.281228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b5d89-3512-4beb-a919-5105e655d41f None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:53.282571 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 15:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:53.287938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2913a8-1a91-4220-a771-7092e24cb138 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:53.288343 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2913a8-1a91-4220-a771-7092e24cb138 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:53.288490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2913a8-1a91-4220-a771-7092e24cb138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:53.354031 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-6d2913a8-1a91-4220-a771-7092e24cb138 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:53.355241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-6d2913a8-1a91-4220-a771-7092e24cb138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:53.368306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:53.455516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 15:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:53.493522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b5d89-3512-4beb-a919-5105e655d41f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:53.495090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-679b5d89-3512-4beb-a919-5105e655d41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:53.600322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b5d89-3512-4beb-a919-5105e655d41f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:53.612331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:53.673927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0eb23d-e57b-45ff-b594-9e398ef3678a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:53.674577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0eb23d-e57b-45ff-b594-9e398ef3678a service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:53.674865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0eb23d-e57b-45ff-b594-9e398ef3678a service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:53.675148 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0eb23d-e57b-45ff-b594-9e398ef3678a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:53.676803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-df0eb23d-e57b-45ff-b594-9e398ef3678a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:53.790350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0eb23d-e57b-45ff-b594-9e398ef3678a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:53.791805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0eb23d-e57b-45ff-b594-9e398ef3678a service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:53.809777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 714/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:56.171062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad195588-31f5-451b-a9c0-2a5eadf4f62e None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:56.171633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad195588-31f5-451b-a9c0-2a5eadf4f62e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:56.171633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad195588-31f5-451b-a9c0-2a5eadf4f62e None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:56.172997 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 15:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:56.178575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec760c-4353-48a9-90d2-0590caa10a5d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:56.179000 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec760c-4353-48a9-90d2-0590caa10a5d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:56.179142 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec760c-4353-48a9-90d2-0590caa10a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:56.226935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:56.245564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-73ec760c-4353-48a9-90d2-0590caa10a5d None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:56.246765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-73ec760c-4353-48a9-90d2-0590caa10a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:56.343672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 15:23:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:56.382225 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad195588-31f5-451b-a9c0-2a5eadf4f62e None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:56.383739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ad195588-31f5-451b-a9c0-2a5eadf4f62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:56.485790 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad195588-31f5-451b-a9c0-2a5eadf4f62e None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:56.525505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:56.559140 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b78aa-9a58-45f3-9532-bbd6192f3fd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:56.559889 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b78aa-9a58-45f3-9532-bbd6192f3fd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:56.560174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b78aa-9a58-45f3-9532-bbd6192f3fd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:56.560450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b78aa-9a58-45f3-9532-bbd6192f3fd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:56.562161 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-680b78aa-9a58-45f3-9532-bbd6192f3fd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:56.669038 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b78aa-9a58-45f3-9532-bbd6192f3fd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:56.670832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b78aa-9a58-45f3-9532-bbd6192f3fd8 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:56.688466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 734/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:23:58.884274 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06708034-9127-4ee2-9e74-f7324003e429 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:58.884727 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06708034-9127-4ee2-9e74-f7324003e429 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:58.884824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06708034-9127-4ee2-9e74-f7324003e429 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:58.886246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 15:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:23:58.892154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09082918-e8f5-49d9-8ed0-1dc1523f68b3 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:58.892838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09082918-e8f5-49d9-8ed0-1dc1523f68b3 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:58.892978 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09082918-e8f5-49d9-8ed0-1dc1523f68b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:58.957240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-09082918-e8f5-49d9-8ed0-1dc1523f68b3 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:23:58.958457 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-09082918-e8f5-49d9-8ed0-1dc1523f68b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:58.979941 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:59.055481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 15:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:23:59.094931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06708034-9127-4ee2-9e74-f7324003e429 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:23:59.096496 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-06708034-9127-4ee2-9e74-f7324003e429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:59.200750 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06708034-9127-4ee2-9e74-f7324003e429 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:23:59.224400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:23:59.280230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65c771-44e0-4ed2-8065-70b6d148432b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:23:59.280877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65c771-44e0-4ed2-8065-70b6d148432b service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:23:59.281188 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65c771-44e0-4ed2-8065-70b6d148432b service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:23:59.281470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65c771-44e0-4ed2-8065-70b6d148432b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:23:59.283111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ee65c771-44e0-4ed2-8065-70b6d148432b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:23:59.392023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65c771-44e0-4ed2-8065-70b6d148432b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:23:59.394344 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65c771-44e0-4ed2-8065-70b6d148432b service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:23:59.412378 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 717/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:23:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:24:01.485225 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056a523-0969-4a14-a178-01ce31f21285 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:01.485793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056a523-0969-4a14-a178-01ce31f21285 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:01.485793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056a523-0969-4a14-a178-01ce31f21285 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:01.487142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 15:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:24:01.492522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68470871-31d0-41cc-abbb-554c418e6006 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:01.492954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68470871-31d0-41cc-abbb-554c418e6006 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:01.493058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68470871-31d0-41cc-abbb-554c418e6006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:01.556792 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:01.560223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-68470871-31d0-41cc-abbb-554c418e6006 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:24:01.562012 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-68470871-31d0-41cc-abbb-554c418e6006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:01.659024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 15:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:24:01.697929 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056a523-0969-4a14-a178-01ce31f21285 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:24:01.699458 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5056a523-0969-4a14-a178-01ce31f21285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:01.802274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056a523-0969-4a14-a178-01ce31f21285 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:24:01.853647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:01.877907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15d6c8-a98a-4ab3-8a1f-9e8bf5ea9264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:24:01.878600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15d6c8-a98a-4ab3-8a1f-9e8bf5ea9264 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:01.878883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15d6c8-a98a-4ab3-8a1f-9e8bf5ea9264 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:01.879155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15d6c8-a98a-4ab3-8a1f-9e8bf5ea9264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:01.880837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ad15d6c8-a98a-4ab3-8a1f-9e8bf5ea9264 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:01.988374 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15d6c8-a98a-4ab3-8a1f-9e8bf5ea9264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:24:01.989843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15d6c8-a98a-4ab3-8a1f-9e8bf5ea9264 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:24:02.008922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 737/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:24:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:24:04.125194 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0117b-231f-468a-a7b1-6ded41f242a2 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:04.125733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0117b-231f-468a-a7b1-6ded41f242a2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:04.125733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0117b-231f-468a-a7b1-6ded41f242a2 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:04.127139 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 15:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:24:04.132575 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c6f42-e580-4365-800b-64da4d0e080d None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:04.133009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c6f42-e580-4365-800b-64da4d0e080d None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:04.133187 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c6f42-e580-4365-800b-64da4d0e080d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:04.198692 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-eb4c6f42-e580-4365-800b-64da4d0e080d None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:24:04.199895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-eb4c6f42-e580-4365-800b-64da4d0e080d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:04.228724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:04.296174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 15:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:24:04.334908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0117b-231f-468a-a7b1-6ded41f242a2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:24:04.336433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-39a0117b-231f-468a-a7b1-6ded41f242a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:04.439276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0117b-231f-468a-a7b1-6ded41f242a2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:24:04.467622 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:04.511499 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa089ee-6c46-4637-99d0-7d52ac75d89f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:24:04.512143 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa089ee-6c46-4637-99d0-7d52ac75d89f service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:04.512425 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa089ee-6c46-4637-99d0-7d52ac75d89f service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:04.512697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa089ee-6c46-4637-99d0-7d52ac75d89f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:04.514919 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1fa089ee-6c46-4637-99d0-7d52ac75d89f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:04.625044 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa089ee-6c46-4637-99d0-7d52ac75d89f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:24:04.626505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa089ee-6c46-4637-99d0-7d52ac75d89f service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:24:04.647170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 720/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:24:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:24:06.742972 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ee701-6fd2-43d1-96f1-78a7db7cd34a None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:06.743525 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ee701-6fd2-43d1-96f1-78a7db7cd34a None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:06.743525 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ee701-6fd2-43d1-96f1-78a7db7cd34a None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:06.744964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 15:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:24:06.750313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6dd11-ea8b-4e77-841f-d393675e839e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:06.750829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6dd11-ea8b-4e77-841f-d393675e839e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:06.750940 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6dd11-ea8b-4e77-841f-d393675e839e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:06.815577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a5b6dd11-ea8b-4e77-841f-d393675e839e None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:24:06.816781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a5b6dd11-ea8b-4e77-841f-d393675e839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:06.824918 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:06.912815 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 15:24:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:24:06.953873 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ee701-6fd2-43d1-96f1-78a7db7cd34a None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:24:06.955432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-122ee701-6fd2-43d1-96f1-78a7db7cd34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:07.058384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ee701-6fd2-43d1-96f1-78a7db7cd34a None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:24:07.113309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:07.131747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f352-8d09-4789-b5ce-3071c3a109db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:24:07.132395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f352-8d09-4789-b5ce-3071c3a109db service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:07.132679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f352-8d09-4789-b5ce-3071c3a109db service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:07.132958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f352-8d09-4789-b5ce-3071c3a109db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:07.134631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2e18f352-8d09-4789-b5ce-3071c3a109db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:07.244528 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18f352-8d09-4789-b5ce-3071c3a109db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:24:07.246065 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18f352-8d09-4789-b5ce-3071c3a109db service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:24:07.265169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 740/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:24:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:24:09.396341 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce09cb3-3c77-4b9b-a637-e5ba3570c646 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:09.397188 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce09cb3-3c77-4b9b-a637-e5ba3570c646 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:09.397372 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce09cb3-3c77-4b9b-a637-e5ba3570c646 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:09.398872 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 15:24:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:24:09.404288 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c3221-c734-45ec-a302-1628133a0758 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:09.404765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c3221-c734-45ec-a302-1628133a0758 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:09.405948 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c3221-c734-45ec-a302-1628133a0758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:09.471360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-8f9c3221-c734-45ec-a302-1628133a0758 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:24:09.472561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-8f9c3221-c734-45ec-a302-1628133a0758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:09.509803 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:09.569533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 15:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:24:09.607634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce09cb3-3c77-4b9b-a637-e5ba3570c646 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:24:09.609236 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4ce09cb3-3c77-4b9b-a637-e5ba3570c646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:09.713805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce09cb3-3c77-4b9b-a637-e5ba3570c646 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:24:09.751125 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:09.786951 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fb917-830b-4df3-9e1c-bea83d133d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:24:09.787525 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fb917-830b-4df3-9e1c-bea83d133d8b service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:09.787804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fb917-830b-4df3-9e1c-bea83d133d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:09.788076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fb917-830b-4df3-9e1c-bea83d133d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:09.789730 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-814fb917-830b-4df3-9e1c-bea83d133d8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:09.896255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fb917-830b-4df3-9e1c-bea83d133d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:24:09.897746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fb917-830b-4df3-9e1c-bea83d133d8b service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:24:09.915647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 723/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:24:11.945681 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033782d-9530-4b6f-bcc4-ed2af106faf7 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:11.946229 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033782d-9530-4b6f-bcc4-ed2af106faf7 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:11.946229 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033782d-9530-4b6f-bcc4-ed2af106faf7 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:11.947623 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 15:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:24:11.952969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6a1e2-1916-4e01-8855-4cd6dec40b58 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:11.953426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6a1e2-1916-4e01-8855-4cd6dec40b58 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:11.953569 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6a1e2-1916-4e01-8855-4cd6dec40b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:12.019217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e6c6a1e2-1916-4e01-8855-4cd6dec40b58 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:24:12.020404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e6c6a1e2-1916-4e01-8855-4cd6dec40b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:12.038685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:12.118146 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 15:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:24:12.155761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033782d-9530-4b6f-bcc4-ed2af106faf7 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:24:12.157316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e033782d-9530-4b6f-bcc4-ed2af106faf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:12.261706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033782d-9530-4b6f-bcc4-ed2af106faf7 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:24:12.329038 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:12.336083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f88f8-6eb9-4d25-9d1a-657bf70852cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:24:12.336721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f88f8-6eb9-4d25-9d1a-657bf70852cf service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:12.337005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f88f8-6eb9-4d25-9d1a-657bf70852cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:12.337345 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f88f8-6eb9-4d25-9d1a-657bf70852cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:12.338978 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f93f88f8-6eb9-4d25-9d1a-657bf70852cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:12.447992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f88f8-6eb9-4d25-9d1a-657bf70852cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:24:12.449461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f88f8-6eb9-4d25-9d1a-657bf70852cf service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:24:12.468838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 743/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:24:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:24:14.525855 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d53dbe-b612-41cf-9bcf-454a9ccefe64 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:14.526388 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d53dbe-b612-41cf-9bcf-454a9ccefe64 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:14.526388 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d53dbe-b612-41cf-9bcf-454a9ccefe64 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:14.527733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 15:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:24:14.533092 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b7fcb-3b0b-4851-ab81-fb6cdf444f0a None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:14.533540 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b7fcb-3b0b-4851-ab81-fb6cdf444f0a None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:14.533692 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b7fcb-3b0b-4851-ab81-fb6cdf444f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:14.597949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-982b7fcb-3b0b-4851-ab81-fb6cdf444f0a None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:24:14.599156 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-982b7fcb-3b0b-4851-ab81-fb6cdf444f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:14.646462 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:14.694109 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 15:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:24:14.731850 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d53dbe-b612-41cf-9bcf-454a9ccefe64 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:24:14.733648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-78d53dbe-b612-41cf-9bcf-454a9ccefe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:14.840421 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d53dbe-b612-41cf-9bcf-454a9ccefe64 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:24:14.887567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:14.914903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadecb0-c463-4ca5-8016-9adcd12f63b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:24:14.915553 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadecb0-c463-4ca5-8016-9adcd12f63b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:14.915844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadecb0-c463-4ca5-8016-9adcd12f63b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:14.916126 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadecb0-c463-4ca5-8016-9adcd12f63b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:14.917803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-eeadecb0-c463-4ca5-8016-9adcd12f63b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:15.026309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadecb0-c463-4ca5-8016-9adcd12f63b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:24:15.027754 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadecb0-c463-4ca5-8016-9adcd12f63b1 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:24:15.045677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 726/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:24:24.692342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=22 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:25.043305 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=66 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:34.911584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8cfb9-ff62-4d3a-9194-c2aa52485c13 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:34.912035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8cfb9-ff62-4d3a-9194-c2aa52485c13 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:34.912166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8cfb9-ff62-4d3a-9194-c2aa52485c13 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:34.913564 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 745/1471] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:24:34.918826 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515246bb-1a4f-41fe-b5aa-766b597a104c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:34.919248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515246bb-1a4f-41fe-b5aa-766b597a104c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:34.919509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515246bb-1a4f-41fe-b5aa-766b597a104c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:34.990457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-515246bb-1a4f-41fe-b5aa-766b597a104c None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:24:34.991678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-515246bb-1a4f-41fe-b5aa-766b597a104c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:35.091933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 727/1472] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:24:35.177456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df388e33-35c1-46e1-b7de-4daf2375d423 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:35.177740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df388e33-35c1-46e1-b7de-4daf2375d423 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:35.178016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df388e33-35c1-46e1-b7de-4daf2375d423 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:35.179377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 746/1473] 10.222.0.42 () {54 vars in 961 bytes} [Tue Jul 19 15:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:24:35.184080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56914f-d490-456f-ab91-5f261d17ff61 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:35.184365 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56914f-d490-456f-ab91-5f261d17ff61 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:35.184665 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56914f-d490-456f-ab91-5f261d17ff61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:35.258664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8a56914f-d490-456f-ab91-5f261d17ff61 None None] MFA Rules not processed for user `215c6f6f1aad486f85ce20590d889cbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:24:35.259873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8a56914f-d490-456f-ab91-5f261d17ff61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:35.280959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:35.337325 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 728/1474] 10.222.0.42 () {58 vars in 1058 bytes} [Tue Jul 19 15:24:35 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:24:35.345231 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df388e33-35c1-46e1-b7de-4daf2375d423 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:24:35.346742 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-df388e33-35c1-46e1-b7de-4daf2375d423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:35.451170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df388e33-35c1-46e1-b7de-4daf2375d423 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:24:35.524214 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb817b-4ff8-4baa-bbab-9c5a043a1867 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '215c6f6f1aad486f85ce20590d889cbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:24:35.524866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb817b-4ff8-4baa-bbab-9c5a043a1867 service glance] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:35.525172 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb817b-4ff8-4baa-bbab-9c5a043a1867 service glance] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:35.525461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb817b-4ff8-4baa-bbab-9c5a043a1867 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:35.527101 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-dacb817b-4ff8-4baa-bbab-9c5a043a1867 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:35.581803 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:35.636354 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb817b-4ff8-4baa-bbab-9c5a043a1867 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:24:35.637820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb817b-4ff8-4baa-bbab-9c5a043a1867 service glance] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:24:35.657047 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 747/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:24:37.792496 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f80df6-078e-4628-a966-6ac8991c8f4a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:37.793199 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f80df6-078e-4628-a966-6ac8991c8f4a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:37.793199 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f80df6-078e-4628-a966-6ac8991c8f4a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:37.794457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 729/1476] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:24:37.800630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0a6c4-234d-4aa1-beb4-bf514b1a7a6e None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:37.801118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0a6c4-234d-4aa1-beb4-bf514b1a7a6e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:37.801280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0a6c4-234d-4aa1-beb4-bf514b1a7a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:37.864591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-35a0a6c4-234d-4aa1-beb4-bf514b1a7a6e None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:24:37.865823 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-35a0a6c4-234d-4aa1-beb4-bf514b1a7a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:37.961981 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 748/1477] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:24:37.987673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f80df6-078e-4628-a966-6ac8991c8f4a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:24:37.989291 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-79f80df6-078e-4628-a966-6ac8991c8f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:38.113667 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f80df6-078e-4628-a966-6ac8991c8f4a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:24:38.161834 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:38.212250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb419a-c07a-4245-b665-77127b67b778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:24:38.212908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb419a-c07a-4245-b665-77127b67b778 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:38.213228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb419a-c07a-4245-b665-77127b67b778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:38.213501 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb419a-c07a-4245-b665-77127b67b778 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:38.214235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb419a-c07a-4245-b665-77127b67b778 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:24:38.215996 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-ecdb419a-c07a-4245-b665-77127b67b778 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 15:24:38.215996 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) Jul 19 15:24:38.215996 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: Jul 19 15:24:38.217160 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb419a-c07a-4245-b665-77127b67b778 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:24:38.237890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 730/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Jul 19 15:24:37 2022] GET /identity/v3/projects => generated 2553 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:24:40.391269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10488810-f94f-4505-92bf-f1e646dbe735 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:40.391825 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10488810-f94f-4505-92bf-f1e646dbe735 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:40.391825 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10488810-f94f-4505-92bf-f1e646dbe735 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:40.393239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 749/1479] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:24:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:24:40.398788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a020d37-9a96-4ee6-8f85-41924fc3338c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:40.399231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a020d37-9a96-4ee6-8f85-41924fc3338c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:40.399344 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a020d37-9a96-4ee6-8f85-41924fc3338c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:40.464963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2a020d37-9a96-4ee6-8f85-41924fc3338c None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:24:40.466177 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2a020d37-9a96-4ee6-8f85-41924fc3338c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:40.680152 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 731/1480] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:24:40.724473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10488810-f94f-4505-92bf-f1e646dbe735 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:24:40.726006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-10488810-f94f-4505-92bf-f1e646dbe735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:40.735765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:40.868832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10488810-f94f-4505-92bf-f1e646dbe735 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:24:40.947950 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:40.980412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd223143-fae7-4a3a-bffe-7ec87553e7f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:24:40.981090 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd223143-fae7-4a3a-bffe-7ec87553e7f8 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:40.981396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd223143-fae7-4a3a-bffe-7ec87553e7f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:40.981669 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd223143-fae7-4a3a-bffe-7ec87553e7f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:40.983308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-bd223143-fae7-4a3a-bffe-7ec87553e7f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:41.072318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd223143-fae7-4a3a-bffe-7ec87553e7f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:24:41.073937 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd223143-fae7-4a3a-bffe-7ec87553e7f8 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:24:41.091698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 750/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:24:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:24:43.318021 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37a14f-c85c-4928-8868-560a6d582e80 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:43.318598 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37a14f-c85c-4928-8868-560a6d582e80 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:43.318598 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37a14f-c85c-4928-8868-560a6d582e80 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:43.320018 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 732/1482] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:24:43.325092 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979ff55-b1af-4feb-b572-2816770395d7 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:43.325571 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979ff55-b1af-4feb-b572-2816770395d7 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:43.325680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979ff55-b1af-4feb-b572-2816770395d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:43.391812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-f979ff55-b1af-4feb-b572-2816770395d7 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:24:43.393026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f979ff55-b1af-4feb-b572-2816770395d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:43.440171 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:43.491209 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 751/1483] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:24:43.528604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37a14f-c85c-4928-8868-560a6d582e80 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:24:43.530134 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fb37a14f-c85c-4928-8868-560a6d582e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:43.552979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:43.631541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37a14f-c85c-4928-8868-560a6d582e80 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:24:43.703584 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d6494-1e1e-4ec7-a22f-cae90dc0fd85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:24:43.704246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d6494-1e1e-4ec7-a22f-cae90dc0fd85 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:43.704531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d6494-1e1e-4ec7-a22f-cae90dc0fd85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:43.704805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d6494-1e1e-4ec7-a22f-cae90dc0fd85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:43.706467 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0c6d6494-1e1e-4ec7-a22f-cae90dc0fd85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:43.779595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:43.816470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d6494-1e1e-4ec7-a22f-cae90dc0fd85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:24:43.817925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d6494-1e1e-4ec7-a22f-cae90dc0fd85 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:24:43.836830 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 733/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:24:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:24:45.921851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50649f32-4187-40aa-b589-b33d650e7175 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:45.922405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50649f32-4187-40aa-b589-b33d650e7175 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:45.922405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50649f32-4187-40aa-b589-b33d650e7175 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:45.923699 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 752/1485] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:24:45.928961 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe30b1-f59a-4a12-95bd-15ec148908a7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:45.929401 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe30b1-f59a-4a12-95bd-15ec148908a7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:45.929569 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe30b1-f59a-4a12-95bd-15ec148908a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:45.995333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-fafe30b1-f59a-4a12-95bd-15ec148908a7 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:24:45.996550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fafe30b1-f59a-4a12-95bd-15ec148908a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:46.094575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 734/1486] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:24:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:24:46.134006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50649f32-4187-40aa-b589-b33d650e7175 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:24:46.135556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-50649f32-4187-40aa-b589-b33d650e7175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:46.241951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50649f32-4187-40aa-b589-b33d650e7175 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:24:46.314156 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:46.315995 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85113df-a6f6-40e6-b406-60e6d70faa00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:24:46.316615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85113df-a6f6-40e6-b406-60e6d70faa00 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:46.317085 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85113df-a6f6-40e6-b406-60e6d70faa00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:46.317681 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85113df-a6f6-40e6-b406-60e6d70faa00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:46.319504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f85113df-a6f6-40e6-b406-60e6d70faa00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:46.427631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85113df-a6f6-40e6-b406-60e6d70faa00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:24:46.429072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85113df-a6f6-40e6-b406-60e6d70faa00 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:24:46.447724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 753/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:24:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:24:48.604572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc0dd1-a893-4cfa-a274-5ccae07aaf29 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:48.605184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc0dd1-a893-4cfa-a274-5ccae07aaf29 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:48.605184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc0dd1-a893-4cfa-a274-5ccae07aaf29 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:48.606536 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 735/1488] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:24:48.612183 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803babe-ae5a-4584-a600-ab949f4f53cd None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:48.612625 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803babe-ae5a-4584-a600-ab949f4f53cd None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:48.612757 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803babe-ae5a-4584-a600-ab949f4f53cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:48.675915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-5803babe-ae5a-4584-a600-ab949f4f53cd None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:24:48.677108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5803babe-ae5a-4584-a600-ab949f4f53cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:48.728441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:48.772331 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 754/1489] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:24:48.813182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc0dd1-a893-4cfa-a274-5ccae07aaf29 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:24:48.814695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-49fc0dd1-a893-4cfa-a274-5ccae07aaf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:48.850692 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:48.918722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc0dd1-a893-4cfa-a274-5ccae07aaf29 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:24:48.995949 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32270d9e-51fb-4bbb-a6f8-657c2ac6104d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:24:48.996631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32270d9e-51fb-4bbb-a6f8-657c2ac6104d service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:48.996917 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32270d9e-51fb-4bbb-a6f8-657c2ac6104d service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:48.997218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32270d9e-51fb-4bbb-a6f8-657c2ac6104d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:48.998876 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-32270d9e-51fb-4bbb-a6f8-657c2ac6104d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:49.079255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:49.109281 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32270d9e-51fb-4bbb-a6f8-657c2ac6104d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:24:49.110707 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32270d9e-51fb-4bbb-a6f8-657c2ac6104d service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:24:49.128099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 736/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:24:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:24:51.459870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f6d77-e677-4ba8-b4ff-3fd16003b6f2 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:51.460416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f6d77-e677-4ba8-b4ff-3fd16003b6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:51.460416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f6d77-e677-4ba8-b4ff-3fd16003b6f2 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:51.461747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 755/1491] 10.222.0.42 () {54 vars in 926 bytes} [Tue Jul 19 15:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:24:51.467232 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35f2cd-a55a-4fb2-b283-23c8c560e62e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:51.467681 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35f2cd-a55a-4fb2-b283-23c8c560e62e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:51.467849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35f2cd-a55a-4fb2-b283-23c8c560e62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:51.533459 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0a35f2cd-a55a-4fb2-b283-23c8c560e62e None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:24:51.534669 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0a35f2cd-a55a-4fb2-b283-23c8c560e62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:51.630906 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 737/1492] 10.222.0.42 () {58 vars in 1023 bytes} [Tue Jul 19 15:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:24:51.670800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f6d77-e677-4ba8-b4ff-3fd16003b6f2 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:24:51.672964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-4b1f6d77-e677-4ba8-b4ff-3fd16003b6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:51.776007 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f6d77-e677-4ba8-b4ff-3fd16003b6f2 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:24:51.848405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17ae29-07bd-4ed7-9eb0-1f2a418d7932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:24:51.849051 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17ae29-07bd-4ed7-9eb0-1f2a418d7932 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:24:51.849358 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17ae29-07bd-4ed7-9eb0-1f2a418d7932 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:24:51.849633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17ae29-07bd-4ed7-9eb0-1f2a418d7932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:24:51.851267 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2f17ae29-07bd-4ed7-9eb0-1f2a418d7932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:24:51.860376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:24:51.961389 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17ae29-07bd-4ed7-9eb0-1f2a418d7932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:24:51.962834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17ae29-07bd-4ed7-9eb0-1f2a418d7932 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:24:51.980428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 756/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:24:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:25:01.629158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=82 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:25:01.978099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=50 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:26:25.779759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccf74c-ce50-426c-a7e4-4a39c60eda70 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:25.780334 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccf74c-ce50-426c-a7e4-4a39c60eda70 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:25.780334 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccf74c-ce50-426c-a7e4-4a39c60eda70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:25.855451 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4fccf74c-ce50-426c-a7e4-4a39c60eda70 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:26:25.856642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4fccf74c-ce50-426c-a7e4-4a39c60eda70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:25.950571 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 738/1494] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:26:25.959209 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d119c5-ea56-4adf-9c53-9936fd987411 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:25.959721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d119c5-ea56-4adf-9c53-9936fd987411 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:25.959827 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d119c5-ea56-4adf-9c53-9936fd987411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:26.033999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-25d119c5-ea56-4adf-9c53-9936fd987411 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:26:26.035118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-25d119c5-ea56-4adf-9c53-9936fd987411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:26.128559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 757/1495] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:26:26.133191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccf74c-ce50-426c-a7e4-4a39c60eda70 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:26:26.134376 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4fccf74c-ce50-426c-a7e4-4a39c60eda70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:26.146286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccf74c-ce50-426c-a7e4-4a39c60eda70 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:26:26.207604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:26:26.243086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19ece9-aa94-4200-a0e7-5635abd7e8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:26:26.243754 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19ece9-aa94-4200-a0e7-5635abd7e8bc admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:26.244037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19ece9-aa94-4200-a0e7-5635abd7e8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:26.244315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19ece9-aa94-4200-a0e7-5635abd7e8bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:26.245043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19ece9-aa94-4200-a0e7-5635abd7e8bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:26:26.245391 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19ece9-aa94-4200-a0e7-5635abd7e8bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:26:26.246407 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-aa19ece9-aa94-4200-a0e7-5635abd7e8bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 15:26:26.246407 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) Jul 19 15:26:26.246407 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: Jul 19 15:26:26.247292 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19ece9-aa94-4200-a0e7-5635abd7e8bc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:26:26.259366 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 739/1496] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:26:26.268242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d119c5-ea56-4adf-9c53-9936fd987411 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:26:26.277625 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d119c5-ea56-4adf-9c53-9936fd987411 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:26:26.342807 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:26:26.377904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d44a7d-3373-419e-8998-b6d5262e392d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:26:26.378562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d44a7d-3373-419e-8998-b6d5262e392d admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:26.378842 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d44a7d-3373-419e-8998-b6d5262e392d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:26.379114 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d44a7d-3373-419e-8998-b6d5262e392d admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:26.379905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d44a7d-3373-419e-8998-b6d5262e392d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:26:26.381641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: WARNING py.warnings [None req-b7d44a7d-3373-419e-8998-b6d5262e392d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 15:26:26.381641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: warnings.warn(msg) Jul 19 15:26:26.381641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: Jul 19 15:26:26.382829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d44a7d-3373-419e-8998-b6d5262e392d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:26:26.436457 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 758/1497] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:26:26 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:26:26.441224 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19ece9-aa94-4200-a0e7-5635abd7e8bc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:26:26.442740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-aa19ece9-aa94-4200-a0e7-5635abd7e8bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:26.476653 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19ece9-aa94-4200-a0e7-5635abd7e8bc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:26:26.571147 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f847a21-f1b7-4ea5-a0e5-d0594f94064e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:26:26.571807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f847a21-f1b7-4ea5-a0e5-d0594f94064e admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:26.572085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f847a21-f1b7-4ea5-a0e5-d0594f94064e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:26.572357 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f847a21-f1b7-4ea5-a0e5-d0594f94064e admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:26.573049 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f847a21-f1b7-4ea5-a0e5-d0594f94064e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:26:26.574091 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-1f847a21-f1b7-4ea5-a0e5-d0594f94064e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 15:26:26.574091 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) Jul 19 15:26:26.574091 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: Jul 19 15:26:26.575329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f847a21-f1b7-4ea5-a0e5-d0594f94064e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:26:26.615599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 740/1498] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:26:26 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:26:26.619645 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d44a7d-3373-419e-8998-b6d5262e392d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:26:26.621196 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b7d44a7d-3373-419e-8998-b6d5262e392d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:26.685815 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d44a7d-3373-419e-8998-b6d5262e392d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:26:26.769112 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:26:26.780651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6cd62-0201-47ce-9ce2-d92e33b71e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:26:26.781355 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6cd62-0201-47ce-9ce2-d92e33b71e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:26.781635 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6cd62-0201-47ce-9ce2-d92e33b71e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:26.781908 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6cd62-0201-47ce-9ce2-d92e33b71e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:26.782660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6cd62-0201-47ce-9ce2-d92e33b71e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:26:26.783664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: WARNING py.warnings [None req-b6f6cd62-0201-47ce-9ce2-d92e33b71e90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 15:26:26.783664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: warnings.warn(msg) Jul 19 15:26:26.783664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: Jul 19 15:26:26.784540 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6cd62-0201-47ce-9ce2-d92e33b71e90 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:26:26.797256 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 759/1499] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:26:26.801469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f847a21-f1b7-4ea5-a0e5-d0594f94064e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:26:26.802973 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1f847a21-f1b7-4ea5-a0e5-d0594f94064e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:26.814959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f847a21-f1b7-4ea5-a0e5-d0594f94064e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:26:26.839345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:26:26.910125 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ff6bd-f1d5-4f4b-8e90-f25e44709f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:26:26.910708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ff6bd-f1d5-4f4b-8e90-f25e44709f50 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:26.910989 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ff6bd-f1d5-4f4b-8e90-f25e44709f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:26.911261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ff6bd-f1d5-4f4b-8e90-f25e44709f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:26.911983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ff6bd-f1d5-4f4b-8e90-f25e44709f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:26:26.912980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-566ff6bd-f1d5-4f4b-8e90-f25e44709f50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 15:26:26.912980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) Jul 19 15:26:26.912980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: Jul 19 15:26:26.913916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ff6bd-f1d5-4f4b-8e90-f25e44709f50 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:26:26.926518 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 741/1500] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:26:26.930711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6cd62-0201-47ce-9ce2-d92e33b71e90 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:26:26.932228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b6f6cd62-0201-47ce-9ce2-d92e33b71e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:26.964500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6cd62-0201-47ce-9ce2-d92e33b71e90 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:26:27.059994 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d4de6-4114-4753-aef8-b8d0d822d6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:26:27.060879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d4de6-4114-4753-aef8-b8d0d822d6cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:27.061186 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d4de6-4114-4753-aef8-b8d0d822d6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:27.061469 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d4de6-4114-4753-aef8-b8d0d822d6cd admin admin] PATH_INFO: `/v3/projects/d07728962d3e4e26b5e8f0b6ab476dae/users/d150bc61188943058b3d939c1cf5c13b/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:27.083419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d4de6-4114-4753-aef8-b8d0d822d6cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d07728962d3e4e26b5e8f0b6ab476dae, user_id=d150bc61188943058b3d939c1cf5c13b, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:26:27.084448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: WARNING py.warnings [None req-c39d4de6-4114-4753-aef8-b8d0d822d6cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 15:26:27.084448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: warnings.warn(msg) Jul 19 15:26:27.084448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: Jul 19 15:26:27.086433 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d4de6-4114-4753-aef8-b8d0d822d6cd admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:26:27.112751 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 760/1501] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:26:26 2022] PUT /identity/v3/projects/d07728962d3e4e26b5e8f0b6ab476dae/users/d150bc61188943058b3d939c1cf5c13b/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:26:27.125115 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c05d28-04cd-476d-ba1f-74c03a75b33a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:27.125426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c05d28-04cd-476d-ba1f-74c03a75b33a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:27.125706 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c05d28-04cd-476d-ba1f-74c03a75b33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:27.180834 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-59c05d28-04cd-476d-ba1f-74c03a75b33a None None] MFA Rules not processed for user `d150bc61188943058b3d939c1cf5c13b`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:26:27.182017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-59c05d28-04cd-476d-ba1f-74c03a75b33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:27.266676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 742/1502] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:26:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:26:27.279935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f00ed6-c900-4fbb-97eb-8a9cca4388ee None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:27.280227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f00ed6-c900-4fbb-97eb-8a9cca4388ee None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:27.280505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f00ed6-c900-4fbb-97eb-8a9cca4388ee None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:27.282896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 761/1503] 10.222.0.42 () {54 vars in 964 bytes} [Tue Jul 19 15:26:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:26:27.287742 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf95da3-0fcc-4a66-968e-a9bdd389334d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:27.288033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf95da3-0fcc-4a66-968e-a9bdd389334d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:27.288311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf95da3-0fcc-4a66-968e-a9bdd389334d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:27.318574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:26:27.363769 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7bf95da3-0fcc-4a66-968e-a9bdd389334d None None] MFA Rules not processed for user `7245ef27d2454927a01a7cd56a4c27aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:26:27.364976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7bf95da3-0fcc-4a66-968e-a9bdd389334d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:27.440254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 743/1504] 10.222.0.42 () {58 vars in 1061 bytes} [Tue Jul 19 15:26:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:26:27.448009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f00ed6-c900-4fbb-97eb-8a9cca4388ee None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:26:27.449593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-21f00ed6-c900-4fbb-97eb-8a9cca4388ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:27.499400 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:26:27.555970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f00ed6-c900-4fbb-97eb-8a9cca4388ee None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:26:27.627774 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b814c1-c8a4-44ef-82d0-b16c52a37363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:26:27.628425 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b814c1-c8a4-44ef-82d0-b16c52a37363 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:27.628714 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b814c1-c8a4-44ef-82d0-b16c52a37363 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:27.628996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b814c1-c8a4-44ef-82d0-b16c52a37363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:27.630677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-55b814c1-c8a4-44ef-82d0-b16c52a37363 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:27.728520 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b814c1-c8a4-44ef-82d0-b16c52a37363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:26:27.729992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b814c1-c8a4-44ef-82d0-b16c52a37363 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:26:27.733512 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:26:27.748538 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 762/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:26:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:26:27.791242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245535f-9443-4293-8e10-07a980f98c41 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:27.791529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245535f-9443-4293-8e10-07a980f98c41 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:27.791814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245535f-9443-4293-8e10-07a980f98c41 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:27.793123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 744/1506] 10.222.0.42 () {52 vars in 829 bytes} [Tue Jul 19 15:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:26:27.796740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b814c1-c8a4-44ef-82d0-b16c52a37363 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:26:27.798069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-55b814c1-c8a4-44ef-82d0-b16c52a37363 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:27.897488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b814c1-c8a4-44ef-82d0-b16c52a37363 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:26:27.988166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c47f7-9221-4296-b953-cdff1c8fec1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:26:27.989115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c47f7-9221-4296-b953-cdff1c8fec1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:27.989476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c47f7-9221-4296-b953-cdff1c8fec1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:27.989753 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c47f7-9221-4296-b953-cdff1c8fec1d admin admin] PATH_INFO: `/v3/users/d150bc61188943058b3d939c1cf5c13b` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:28.002379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c47f7-9221-4296-b953-cdff1c8fec1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d150bc61188943058b3d939c1cf5c13b)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:26:28.003398 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: WARNING py.warnings [None req-dc7c47f7-9221-4296-b953-cdff1c8fec1d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 15:26:28.003398 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: warnings.warn(msg) Jul 19 15:26:28.003398 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: Jul 19 15:26:28.004597 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c47f7-9221-4296-b953-cdff1c8fec1d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:26:28.014151 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:26:28.067996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dc7c47f7-9221-4296-b953-cdff1c8fec1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd150bc61188943058b3d939c1cf5c13b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:26:28.085929 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dc7c47f7-9221-4296-b953-cdff1c8fec1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd150bc61188943058b3d939c1cf5c13b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:26:28.086379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dc7c47f7-9221-4296-b953-cdff1c8fec1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd150bc61188943058b3d939c1cf5c13b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:26:28.095392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dc7c47f7-9221-4296-b953-cdff1c8fec1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd150bc61188943058b3d939c1cf5c13b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:26:28.095914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dc7c47f7-9221-4296-b953-cdff1c8fec1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd150bc61188943058b3d939c1cf5c13b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:26:28.114980 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 763/1507] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:26:27 2022] DELETE /identity/v3/users/d150bc61188943058b3d939c1cf5c13b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:26:28.119439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d52d93-8d6a-423d-8a68-52b19dc2da03 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:28.119905 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d52d93-8d6a-423d-8a68-52b19dc2da03 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:28.120008 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d52d93-8d6a-423d-8a68-52b19dc2da03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:28.175585 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b6d52d93-8d6a-423d-8a68-52b19dc2da03 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:26:28.176780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b6d52d93-8d6a-423d-8a68-52b19dc2da03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:28.254045 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:26:28.267150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 745/1508] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:26:28.277855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c47f7-9221-4296-b953-cdff1c8fec1d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:26:28.279073 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-dc7c47f7-9221-4296-b953-cdff1c8fec1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:28.380642 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c47f7-9221-4296-b953-cdff1c8fec1d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:26:28.435824 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:26:28.456227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed3e07-dd75-424a-b572-c38f8f8f8ccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:26:28.456888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed3e07-dd75-424a-b572-c38f8f8f8ccc service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:28.457201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed3e07-dd75-424a-b572-c38f8f8f8ccc service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:28.457482 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed3e07-dd75-424a-b572-c38f8f8f8ccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:28.459126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-86ed3e07-dd75-424a-b572-c38f8f8f8ccc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:28.565885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed3e07-dd75-424a-b572-c38f8f8f8ccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:26:28.567256 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed3e07-dd75-424a-b572-c38f8f8f8ccc service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:26:28.585064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 764/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:26:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:26:28.932878 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d52d93-8d6a-423d-8a68-52b19dc2da03 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:26:28.934189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b6d52d93-8d6a-423d-8a68-52b19dc2da03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:26:28.945723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d52d93-8d6a-423d-8a68-52b19dc2da03 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:26:29.045355 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:26:29.046213 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:26:29.046898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:26:29.046898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4 admin admin] PATH_INFO: `/v3/projects/d07728962d3e4e26b5e8f0b6ab476dae` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:26:29.056654 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d07728962d3e4e26b5e8f0b6ab476dae)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:26:29.057697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 15:26:29.057697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) Jul 19 15:26:29.057697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: Jul 19 15:26:29.058882 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:26:29.117514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd07728962d3e4e26b5e8f0b6ab476dae', 'request_id': 'req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:26:29.127005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd07728962d3e4e26b5e8f0b6ab476dae', 'request_id': 'req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:26:29.133229 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd07728962d3e4e26b5e8f0b6ab476dae', 'request_id': 'req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:26:29.164942 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4 admin admin] The token cache is being invalidate because project d07728962d3e4e26b5e8f0b6ab476dae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:26:29.165258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:26:29.165686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7c2be9e6-b2fa-4448-a627-fc2cb1f216c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:26:29.167164 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 746/1510] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:26:28 2022] DELETE /identity/v3/projects/d07728962d3e4e26b5e8f0b6ab476dae => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:26:38.582911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=62 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:26:39.162965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:28:12.522825 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eeb8d-3e95-4b25-9b96-a71d0626d5d9 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:12.523384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eeb8d-3e95-4b25-9b96-a71d0626d5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:12.523384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eeb8d-3e95-4b25-9b96-a71d0626d5d9 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:12.524691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 15:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:28:12.530661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297e6cf-05f7-4365-aceb-405720736130 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:12.531106 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297e6cf-05f7-4365-aceb-405720736130 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:12.531209 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297e6cf-05f7-4365-aceb-405720736130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:12.612706 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5297e6cf-05f7-4365-aceb-405720736130 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:28:12.613950 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5297e6cf-05f7-4365-aceb-405720736130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:28:12.719195 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 15:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:28:12.876811 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89eeb8d-3e95-4b25-9b96-a71d0626d5d9 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:28:12.879013 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f89eeb8d-3e95-4b25-9b96-a71d0626d5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:28:12.995865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89eeb8d-3e95-4b25-9b96-a71d0626d5d9 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:28:13.001905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8065d-005b-4119-811f-6a60864fc97f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:28:13.002555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8065d-005b-4119-811f-6a60864fc97f service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:13.002837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8065d-005b-4119-811f-6a60864fc97f service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:13.003114 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8065d-005b-4119-811f-6a60864fc97f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:13.011313 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8065d-005b-4119-811f-6a60864fc97f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:28:13.012903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8065d-005b-4119-811f-6a60864fc97f service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:28:13.024015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 766/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:28:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:28:13.864959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38728bf5-26f8-41e7-9c49-f0b88b2f2685 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:13.865607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38728bf5-26f8-41e7-9c49-f0b88b2f2685 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:13.865607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38728bf5-26f8-41e7-9c49-f0b88b2f2685 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:13.866925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 15:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:28:13.871744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63da9e3-ad8c-41f8-b52e-85614eca9082 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:13.872189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63da9e3-ad8c-41f8-b52e-85614eca9082 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:13.872298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63da9e3-ad8c-41f8-b52e-85614eca9082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:13.894178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-a63da9e3-ad8c-41f8-b52e-85614eca9082 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:28:13.895416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a63da9e3-ad8c-41f8-b52e-85614eca9082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:28:13.903434 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 15:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:28:14.059066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38728bf5-26f8-41e7-9c49-f0b88b2f2685 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:28:14.073826 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38728bf5-26f8-41e7-9c49-f0b88b2f2685 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:28:14.079983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff19286c-2c57-4b60-896f-fbfa83da0d77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:28:14.080664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19286c-2c57-4b60-896f-fbfa83da0d77 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:14.080814 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6e4b9-d28f-44aa-8a79-7985f6f77409 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:14.080936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19286c-2c57-4b60-896f-fbfa83da0d77 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:14.081115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6e4b9-d28f-44aa-8a79-7985f6f77409 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:14.081264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19286c-2c57-4b60-896f-fbfa83da0d77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:14.081431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6e4b9-d28f-44aa-8a79-7985f6f77409 None None] PATH_INFO: `/v3` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:14.082860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 768/1516] 10.222.0.18 () {54 vars in 925 bytes} [Tue Jul 19 15:28:14 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:28:14.088935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f88871-328f-4ad4-b45a-71a3bd288958 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:14.089248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f88871-328f-4ad4-b45a-71a3bd288958 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:14.089528 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f88871-328f-4ad4-b45a-71a3bd288958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:14.093555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff19286c-2c57-4b60-896f-fbfa83da0d77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:28:14.095152 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff19286c-2c57-4b60-896f-fbfa83da0d77 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:28:14.104796 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 749/1517] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:28:14.156965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-87f88871-328f-4ad4-b45a-71a3bd288958 None None] MFA Rules not processed for user `6002f32936944b0db6fd04f3851836db`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:28:14.158171 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-87f88871-328f-4ad4-b45a-71a3bd288958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:28:14.223473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:28:14.243197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 769/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Jul 19 15:28:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:28:18.027722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed855f-bb02-4d5d-b035-5317a21d23b0 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:18.028257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed855f-bb02-4d5d-b035-5317a21d23b0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:18.028257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed855f-bb02-4d5d-b035-5317a21d23b0 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:18.030685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 750/1519] 10.222.0.42 () {54 vars in 961 bytes} [Tue Jul 19 15:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:28:18.035633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9b48c-86cd-4bad-813a-a42569066bf9 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:18.036178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9b48c-86cd-4bad-813a-a42569066bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:18.036363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9b48c-86cd-4bad-813a-a42569066bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:18.054766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-d8d9b48c-86cd-4bad-813a-a42569066bf9 None None] MFA Rules not processed for user `6002f32936944b0db6fd04f3851836db`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:28:18.055987 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d8d9b48c-86cd-4bad-813a-a42569066bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:28:18.065444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 770/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Tue Jul 19 15:28:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:28:18.072971 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed855f-bb02-4d5d-b035-5317a21d23b0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:28:18.087502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed855f-bb02-4d5d-b035-5317a21d23b0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:28:18.093570 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9e201-759f-45a6-a7e7-79a46dd9932f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:28:18.094233 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9e201-759f-45a6-a7e7-79a46dd9932f service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:18.094520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9e201-759f-45a6-a7e7-79a46dd9932f service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:18.094798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9e201-759f-45a6-a7e7-79a46dd9932f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:18.103407 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9e201-759f-45a6-a7e7-79a46dd9932f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:28:18.105005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9e201-759f-45a6-a7e7-79a46dd9932f service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:28:18.115429 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 751/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:28:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:28:25.338621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7e56c-a7ff-44a5-9afb-d1f7daefae91 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:25.339173 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7e56c-a7ff-44a5-9afb-d1f7daefae91 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:25.339173 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7e56c-a7ff-44a5-9afb-d1f7daefae91 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:25.340458 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 15:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:28:25.345363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8eb1f3-e464-49d8-b8ba-5d2c20163fee None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:25.345793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8eb1f3-e464-49d8-b8ba-5d2c20163fee None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:25.345932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8eb1f3-e464-49d8-b8ba-5d2c20163fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:25.363846 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bf8eb1f3-e464-49d8-b8ba-5d2c20163fee None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:28:25.365050 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bf8eb1f3-e464-49d8-b8ba-5d2c20163fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:28:25.373526 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:28:25.530943 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7e56c-a7ff-44a5-9afb-d1f7daefae91 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:28:25.545853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7e56c-a7ff-44a5-9afb-d1f7daefae91 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:28:25.551317 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150bf76-6b25-4342-abe5-b02187dc7ae6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:28:25.551957 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150bf76-6b25-4342-abe5-b02187dc7ae6 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:25.552242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150bf76-6b25-4342-abe5-b02187dc7ae6 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:25.552520 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150bf76-6b25-4342-abe5-b02187dc7ae6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:25.561219 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150bf76-6b25-4342-abe5-b02187dc7ae6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:28:25.562817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150bf76-6b25-4342-abe5-b02187dc7ae6 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:28:25.572199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 772/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:28:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:28:26.390326 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34fc1f-f6e8-442e-b789-22a94b6cedc2 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:26.390904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34fc1f-f6e8-442e-b789-22a94b6cedc2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:26.390904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34fc1f-f6e8-442e-b789-22a94b6cedc2 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:26.392287 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 15:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:28:26.397125 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f13c5-2a55-4741-8806-4bc7d0fcdbc5 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:26.397567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f13c5-2a55-4741-8806-4bc7d0fcdbc5 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:26.397712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f13c5-2a55-4741-8806-4bc7d0fcdbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:26.415870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-399f13c5-2a55-4741-8806-4bc7d0fcdbc5 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:28:26.417080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-399f13c5-2a55-4741-8806-4bc7d0fcdbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:28:26.425659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 15:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:28:26.577837 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34fc1f-f6e8-442e-b789-22a94b6cedc2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:28:26.592803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34fc1f-f6e8-442e-b789-22a94b6cedc2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:28:26.598329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e48730-60bd-43a4-b41d-6c92ebe0f2b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:28:26.598967 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e48730-60bd-43a4-b41d-6c92ebe0f2b5 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:26.599250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e48730-60bd-43a4-b41d-6c92ebe0f2b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:26.599528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e48730-60bd-43a4-b41d-6c92ebe0f2b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:26.607778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e48730-60bd-43a4-b41d-6c92ebe0f2b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:28:26.609396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e48730-60bd-43a4-b41d-6c92ebe0f2b5 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:28:26.618711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 754/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:28:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:28:30.697221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef10be3-78f8-448e-b0b4-184b2cf5c0d8 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:30.697766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef10be3-78f8-448e-b0b4-184b2cf5c0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:30.697766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef10be3-78f8-448e-b0b4-184b2cf5c0d8 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:30.699073 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 15:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:28:30.704948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b17ee-4980-469d-bc79-89c5643f5de6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:30.705395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b17ee-4980-469d-bc79-89c5643f5de6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:30.705534 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b17ee-4980-469d-bc79-89c5643f5de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:30.723275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b19b17ee-4980-469d-bc79-89c5643f5de6 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:28:30.724768 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b19b17ee-4980-469d-bc79-89c5643f5de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:28:30.736524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 15:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:28:30.813205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef10be3-78f8-448e-b0b4-184b2cf5c0d8 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:28:30.814953 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9ef10be3-78f8-448e-b0b4-184b2cf5c0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:28:30.893605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef10be3-78f8-448e-b0b4-184b2cf5c0d8 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:28:30.897914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a9041-9927-4e9e-88b9-25dd3419c59f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:28:30.898979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a9041-9927-4e9e-88b9-25dd3419c59f service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:30.899461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a9041-9927-4e9e-88b9-25dd3419c59f service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:30.899934 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a9041-9927-4e9e-88b9-25dd3419c59f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:30.911242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a9041-9927-4e9e-88b9-25dd3419c59f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:28:30.912679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a9041-9927-4e9e-88b9-25dd3419c59f service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:28:30.923982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 775/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:28:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:28:32.963885 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6408b6b-0ce3-49a1-af2f-bc26ba1c05b3 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:32.964421 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6408b6b-0ce3-49a1-af2f-bc26ba1c05b3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:32.964421 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6408b6b-0ce3-49a1-af2f-bc26ba1c05b3 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:32.965882 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 15:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:28:32.971113 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51bbcd-e08c-442c-9b20-f4419c4b5195 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:32.971527 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51bbcd-e08c-442c-9b20-f4419c4b5195 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:32.971675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51bbcd-e08c-442c-9b20-f4419c4b5195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:32.990358 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-2d51bbcd-e08c-442c-9b20-f4419c4b5195 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:28:32.991561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2d51bbcd-e08c-442c-9b20-f4419c4b5195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:28:33.000086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 15:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:28:33.036096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6408b6b-0ce3-49a1-af2f-bc26ba1c05b3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:28:33.047519 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6408b6b-0ce3-49a1-af2f-bc26ba1c05b3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:28:33.051116 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80a34e-1917-4b8e-8919-f6640bd6fb6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:28:33.051772 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80a34e-1917-4b8e-8919-f6640bd6fb6d service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:33.052063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80a34e-1917-4b8e-8919-f6640bd6fb6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:33.052346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80a34e-1917-4b8e-8919-f6640bd6fb6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:33.061567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80a34e-1917-4b8e-8919-f6640bd6fb6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:28:33.063065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80a34e-1917-4b8e-8919-f6640bd6fb6d service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:28:33.072664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 757/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:28:36.621855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f993844-6177-4a7c-8431-278be68cf726 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:36.622562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f993844-6177-4a7c-8431-278be68cf726 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:36.622562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f993844-6177-4a7c-8431-278be68cf726 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:36.623715 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 15:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:28:36.628975 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2333a-c267-44e5-87c0-4e32fbb16d0b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:36.629449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2333a-c267-44e5-87c0-4e32fbb16d0b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:36.629547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2333a-c267-44e5-87c0-4e32fbb16d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:36.635798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:28:36.649427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4ee2333a-c267-44e5-87c0-4e32fbb16d0b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:28:36.650876 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4ee2333a-c267-44e5-87c0-4e32fbb16d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:28:36.659808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 15:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:28:36.965447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f993844-6177-4a7c-8431-278be68cf726 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:28:36.967348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5f993844-6177-4a7c-8431-278be68cf726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:28:37.045207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f993844-6177-4a7c-8431-278be68cf726 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:28:37.048702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d8965-6495-4a72-9b1a-e4594a816c98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:28:37.049372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d8965-6495-4a72-9b1a-e4594a816c98 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:37.049657 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d8965-6495-4a72-9b1a-e4594a816c98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:37.049932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d8965-6495-4a72-9b1a-e4594a816c98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:37.058823 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d8965-6495-4a72-9b1a-e4594a816c98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:28:37.060772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d8965-6495-4a72-9b1a-e4594a816c98 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:28:37.070059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 778/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:28:39.163804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe1c00-4254-4601-90fa-03e9a456f138 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:39.164410 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe1c00-4254-4601-90fa-03e9a456f138 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:39.164410 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe1c00-4254-4601-90fa-03e9a456f138 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:39.165768 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 15:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:28:39.171725 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac1bec-6780-4762-954d-6314a32665ff None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:39.172131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac1bec-6780-4762-954d-6314a32665ff None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:39.172289 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac1bec-6780-4762-954d-6314a32665ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:39.190475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-e5ac1bec-6780-4762-954d-6314a32665ff None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:28:39.191698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e5ac1bec-6780-4762-954d-6314a32665ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:28:39.199561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 15:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:28:39.506622 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe1c00-4254-4601-90fa-03e9a456f138 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:28:39.508261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-abbe1c00-4254-4601-90fa-03e9a456f138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:28:39.521948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe1c00-4254-4601-90fa-03e9a456f138 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:28:39.525655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0bd17-0cce-4b91-8a51-9c86a52d943f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:28:39.526284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0bd17-0cce-4b91-8a51-9c86a52d943f service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:28:39.526564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0bd17-0cce-4b91-8a51-9c86a52d943f service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:28:39.526837 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0bd17-0cce-4b91-8a51-9c86a52d943f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:28:39.536240 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0bd17-0cce-4b91-8a51-9c86a52d943f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:28:39.537689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0bd17-0cce-4b91-8a51-9c86a52d943f service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:28:39.548072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 760/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:28:49.185163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=92 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:28:49.543750 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=8 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:09.962109 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98620a8-1764-4540-9e3a-4ea939a2d7d1 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:09.962718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1458e-27eb-4759-a4c3-6e36d1a6662c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:09.963702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98620a8-1764-4540-9e3a-4ea939a2d7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:09.963702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98620a8-1764-4540-9e3a-4ea939a2d7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:09.964444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1458e-27eb-4759-a4c3-6e36d1a6662c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:09.964966 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1458e-27eb-4759-a4c3-6e36d1a6662c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.002001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-99d1458e-27eb-4759-a4c3-6e36d1a6662c None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.010474 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-a98620a8-1764-4540-9e3a-4ea939a2d7d1 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.011559 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-99d1458e-27eb-4759-a4c3-6e36d1a6662c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.014030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a98620a8-1764-4540-9e3a-4ea939a2d7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.031687 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 761/1540] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:30:10.033016 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfda45-e17b-404a-839a-01fb9fb8842e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.033341 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfda45-e17b-404a-839a-01fb9fb8842e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.033623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfda45-e17b-404a-839a-01fb9fb8842e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.037187 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 780/1541] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:10.049344 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92835297-2c8f-43fc-9727-2c33e1893a6e None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.049667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92835297-2c8f-43fc-9727-2c33e1893a6e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.049918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92835297-2c8f-43fc-9727-2c33e1893a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.067930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-dabfda45-e17b-404a-839a-01fb9fb8842e None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.069181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-dabfda45-e17b-404a-839a-01fb9fb8842e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.078197 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 762/1542] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:10.079529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350d9e2-2972-4c13-aadd-30283c82c483 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.079826 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350d9e2-2972-4c13-aadd-30283c82c483 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.080110 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350d9e2-2972-4c13-aadd-30283c82c483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.082643 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-92835297-2c8f-43fc-9727-2c33e1893a6e None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.083986 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-92835297-2c8f-43fc-9727-2c33e1893a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.102527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b350d9e2-2972-4c13-aadd-30283c82c483 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.103789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b350d9e2-2972-4c13-aadd-30283c82c483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.113365 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 763/1543] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:10.114435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b076ebd-399a-456a-b06f-5b56d6cd3f23 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.114728 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b076ebd-399a-456a-b06f-5b56d6cd3f23 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.114868 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 781/1544] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:10.115059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b076ebd-399a-456a-b06f-5b56d6cd3f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.116230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34529e7f-f310-4098-8655-d3fa31bb389e None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.116520 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34529e7f-f310-4098-8655-d3fa31bb389e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.116793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34529e7f-f310-4098-8655-d3fa31bb389e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.135776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8b076ebd-399a-456a-b06f-5b56d6cd3f23 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.137190 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8b076ebd-399a-456a-b06f-5b56d6cd3f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.149129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 764/1545] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:10.150571 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b076ebd-399a-456a-b06f-5b56d6cd3f23 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.156459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-34529e7f-f310-4098-8655-d3fa31bb389e None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.157703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-34529e7f-f310-4098-8655-d3fa31bb389e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.163807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b076ebd-399a-456a-b06f-5b56d6cd3f23 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.166119 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 782/1546] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:10.169183 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34529e7f-f310-4098-8655-d3fa31bb389e None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.169350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa72c85-e84a-44d0-916a-33c74c03ba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.169739 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa72c85-e84a-44d0-916a-33c74c03ba76 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.170031 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa72c85-e84a-44d0-916a-33c74c03ba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.170297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa72c85-e84a-44d0-916a-33c74c03ba76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.171023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa72c85-e84a-44d0-916a-33c74c03ba76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:10.171329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa72c85-e84a-44d0-916a-33c74c03ba76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.172935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa72c85-e84a-44d0-916a-33c74c03ba76 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.177838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34529e7f-f310-4098-8655-d3fa31bb389e None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.183024 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303eacaf-640c-46ea-ac96-91cb2905e814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.183676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303eacaf-640c-46ea-ac96-91cb2905e814 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.183959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303eacaf-640c-46ea-ac96-91cb2905e814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.184235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303eacaf-640c-46ea-ac96-91cb2905e814 admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.185025 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 765/1547] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:30:10.185178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303eacaf-640c-46ea-ac96-91cb2905e814 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:10.185332 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303eacaf-640c-46ea-ac96-91cb2905e814 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.186282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c089732-11a0-4e80-9a1d-8177d911addd None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.186406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: WARNING py.warnings [None req-303eacaf-640c-46ea-ac96-91cb2905e814 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 15:30:10.186406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: warnings.warn(msg) Jul 19 15:30:10.186406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: Jul 19 15:30:10.186700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c089732-11a0-4e80-9a1d-8177d911addd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.186886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c089732-11a0-4e80-9a1d-8177d911addd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.187260 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303eacaf-640c-46ea-ac96-91cb2905e814 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.201245 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 783/1548] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:30:10.201245 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213cdf8-792d-47e0-bc84-31bc02eb905b None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.201537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213cdf8-792d-47e0-bc84-31bc02eb905b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.201809 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213cdf8-792d-47e0-bc84-31bc02eb905b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.212686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8c089732-11a0-4e80-9a1d-8177d911addd None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.214445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8c089732-11a0-4e80-9a1d-8177d911addd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.219899 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-a213cdf8-792d-47e0-bc84-31bc02eb905b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.221071 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a213cdf8-792d-47e0-bc84-31bc02eb905b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.222448 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 766/1549] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:10.223613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e46e7f-e030-404a-b594-2cc127a78685 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.223900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e46e7f-e030-404a-b594-2cc127a78685 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.224181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e46e7f-e030-404a-b594-2cc127a78685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.229863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 784/1550] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:10.232613 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cce04-53bd-4624-94e7-afdeed683238 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.232982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cce04-53bd-4624-94e7-afdeed683238 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.233374 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cce04-53bd-4624-94e7-afdeed683238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.242769 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c1e46e7f-e030-404a-b594-2cc127a78685 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.243960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c1e46e7f-e030-404a-b594-2cc127a78685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.258920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 767/1551] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:10.259775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-7e6cce04-53bd-4624-94e7-afdeed683238 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.259880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2097e-c514-4bd9-9901-00d753ffd843 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.260545 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7e6cce04-53bd-4624-94e7-afdeed683238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.260640 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2097e-c514-4bd9-9901-00d753ffd843 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.260640 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2097e-c514-4bd9-9901-00d753ffd843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.262822 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 785/1552] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:10.263638 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cce04-53bd-4624-94e7-afdeed683238 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.273438 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cce04-53bd-4624-94e7-afdeed683238 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.273745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-fca2097e-c514-4bd9-9901-00d753ffd843 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.275198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fca2097e-c514-4bd9-9901-00d753ffd843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.279561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade4417-5a2d-42be-b462-6c6679f8d463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.280225 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade4417-5a2d-42be-b462-6c6679f8d463 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.280503 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade4417-5a2d-42be-b462-6c6679f8d463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.280776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade4417-5a2d-42be-b462-6c6679f8d463 admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.281551 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade4417-5a2d-42be-b462-6c6679f8d463 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:10.281970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade4417-5a2d-42be-b462-6c6679f8d463 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.283571 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade4417-5a2d-42be-b462-6c6679f8d463 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.284274 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 768/1553] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:10.285039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2097e-c514-4bd9-9901-00d753ffd843 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.295043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2097e-c514-4bd9-9901-00d753ffd843 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.300222 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3284189-5585-4c7e-8cd5-441576121f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.307114 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 786/1554] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:30:10.307217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3284189-5585-4c7e-8cd5-441576121f48 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.307305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade4417-5a2d-42be-b462-6c6679f8d463 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.307395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3284189-5585-4c7e-8cd5-441576121f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.307395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3284189-5585-4c7e-8cd5-441576121f48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.307395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3284189-5585-4c7e-8cd5-441576121f48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.307395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-a3284189-5585-4c7e-8cd5-441576121f48 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 15:30:10.307395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) Jul 19 15:30:10.307395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: Jul 19 15:30:10.307395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3284189-5585-4c7e-8cd5-441576121f48 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.317351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade4417-5a2d-42be-b462-6c6679f8d463 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.321890 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee530f-9fa7-4ef7-85ae-e1b349b2626a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.323316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee530f-9fa7-4ef7-85ae-e1b349b2626a admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.323678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee530f-9fa7-4ef7-85ae-e1b349b2626a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.323955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee530f-9fa7-4ef7-85ae-e1b349b2626a admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.324950 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee530f-9fa7-4ef7-85ae-e1b349b2626a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.326562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee530f-9fa7-4ef7-85ae-e1b349b2626a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.344637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 769/1555] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:10.344637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa86ff-e71e-4f7c-8577-a973fedd4538 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.344637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa86ff-e71e-4f7c-8577-a973fedd4538 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.344637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa86ff-e71e-4f7c-8577-a973fedd4538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.353578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 787/1556] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:10.355223 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bbc7b-1c93-4ac2-b324-c3884786314e None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.355515 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bbc7b-1c93-4ac2-b324-c3884786314e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.355801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bbc7b-1c93-4ac2-b324-c3884786314e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.363055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7caa86ff-e71e-4f7c-8577-a973fedd4538 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.364249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7caa86ff-e71e-4f7c-8577-a973fedd4538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.373681 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-4b1bbc7b-1c93-4ac2-b324-c3884786314e None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.374874 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-4b1bbc7b-1c93-4ac2-b324-c3884786314e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.424950 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 770/1557] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:10.426409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7a457-2987-4f3c-aeab-2afd451bd207 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.426819 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7a457-2987-4f3c-aeab-2afd451bd207 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.427110 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7a457-2987-4f3c-aeab-2afd451bd207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.435042 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 788/1558] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:10.436349 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf4707-e38e-4f8e-829f-fed7e4fed73f None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.436646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf4707-e38e-4f8e-829f-fed7e4fed73f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.436925 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf4707-e38e-4f8e-829f-fed7e4fed73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.445771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bbf7a457-2987-4f3c-aeab-2afd451bd207 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.447007 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bbf7a457-2987-4f3c-aeab-2afd451bd207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.454919 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-cbaf4707-e38e-4f8e-829f-fed7e4fed73f None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:10.455732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 771/1559] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:10.456087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cbaf4707-e38e-4f8e-829f-fed7e4fed73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:10.456508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7a457-2987-4f3c-aeab-2afd451bd207 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.465527 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 789/1560] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:10.466301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf4707-e38e-4f8e-829f-fed7e4fed73f None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.466606 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7a457-2987-4f3c-aeab-2afd451bd207 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.472997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3df3bf-2baa-4e1f-8a9c-326a49990c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.473681 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3df3bf-2baa-4e1f-8a9c-326a49990c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.473969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3df3bf-2baa-4e1f-8a9c-326a49990c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.474251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3df3bf-2baa-4e1f-8a9c-326a49990c0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.474985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3df3bf-2baa-4e1f-8a9c-326a49990c0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:10.475297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3df3bf-2baa-4e1f-8a9c-326a49990c0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.475636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf4707-e38e-4f8e-829f-fed7e4fed73f None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.476898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3df3bf-2baa-4e1f-8a9c-326a49990c0a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.480515 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98682b-77b7-4e6b-8859-90671d36493c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.481180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98682b-77b7-4e6b-8859-90671d36493c admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.481461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98682b-77b7-4e6b-8859-90671d36493c admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.481737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98682b-77b7-4e6b-8859-90671d36493c admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.482419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea98682b-77b7-4e6b-8859-90671d36493c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.484314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea98682b-77b7-4e6b-8859-90671d36493c admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.488910 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 772/1561] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:30:10.489728 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3df3bf-2baa-4e1f-8a9c-326a49990c0a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.502133 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3df3bf-2baa-4e1f-8a9c-326a49990c0a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.507180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a2d0f-2ffc-48d0-9015-2868b8a22206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.508359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a2d0f-2ffc-48d0-9015-2868b8a22206 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.508485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a2d0f-2ffc-48d0-9015-2868b8a22206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.508686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a2d0f-2ffc-48d0-9015-2868b8a22206 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.509477 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a2d0f-2ffc-48d0-9015-2868b8a22206 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.511992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a2d0f-2ffc-48d0-9015-2868b8a22206 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.518380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 790/1562] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:10.518380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98682b-77b7-4e6b-8859-90671d36493c admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.537307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 773/1563] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/users => generated 290 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:10.537307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a2d0f-2ffc-48d0-9015-2868b8a22206 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.547460 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a2d0f-2ffc-48d0-9015-2868b8a22206 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.578681 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:10.579277 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:10.599162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98682b-77b7-4e6b-8859-90671d36493c admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.604553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a30e1-2eb8-4497-a48e-7445358eb236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.605258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a30e1-2eb8-4497-a48e-7445358eb236 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.605545 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a30e1-2eb8-4497-a48e-7445358eb236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.605826 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a30e1-2eb8-4497-a48e-7445358eb236 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.606523 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a30e1-2eb8-4497-a48e-7445358eb236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.607549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: WARNING py.warnings [None req-308a30e1-2eb8-4497-a48e-7445358eb236 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 15:30:10.607549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: warnings.warn(msg) Jul 19 15:30:10.607549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: Jul 19 15:30:10.608776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a30e1-2eb8-4497-a48e-7445358eb236 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.608874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be2e4d-d7bc-47b1-82db-b403ad2e53d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.609492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be2e4d-d7bc-47b1-82db-b403ad2e53d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.609779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be2e4d-d7bc-47b1-82db-b403ad2e53d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.610062 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be2e4d-d7bc-47b1-82db-b403ad2e53d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.610796 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be2e4d-d7bc-47b1-82db-b403ad2e53d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:10.611110 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be2e4d-d7bc-47b1-82db-b403ad2e53d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.612817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be2e4d-d7bc-47b1-82db-b403ad2e53d6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.634143 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 774/1564] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:30:10.634143 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be2e4d-d7bc-47b1-82db-b403ad2e53d6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.634424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 791/1565] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:10.634424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a30e1-2eb8-4497-a48e-7445358eb236 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.644680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a30e1-2eb8-4497-a48e-7445358eb236 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.644853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be2e4d-d7bc-47b1-82db-b403ad2e53d6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.649309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6a36c-f78a-4c60-a04d-b61a8839555e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.649475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8baab-66bf-44fb-984f-d6722558d9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.650208 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6a36c-f78a-4c60-a04d-b61a8839555e admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.650404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8baab-66bf-44fb-984f-d6722558d9ff admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.650513 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6a36c-f78a-4c60-a04d-b61a8839555e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.650741 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8baab-66bf-44fb-984f-d6722558d9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.650954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6a36c-f78a-4c60-a04d-b61a8839555e admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.651060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8baab-66bf-44fb-984f-d6722558d9ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.651781 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6a36c-f78a-4c60-a04d-b61a8839555e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:10.651886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8baab-66bf-44fb-984f-d6722558d9ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:10.652096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6a36c-f78a-4c60-a04d-b61a8839555e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.652207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8baab-66bf-44fb-984f-d6722558d9ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.653725 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8baab-66bf-44fb-984f-d6722558d9ff admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.653849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6a36c-f78a-4c60-a04d-b61a8839555e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.666115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 792/1566] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:30:10.666529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 775/1567] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:30:10.667384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6a36c-f78a-4c60-a04d-b61a8839555e admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.674334 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8baab-66bf-44fb-984f-d6722558d9ff admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.677355 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6a36c-f78a-4c60-a04d-b61a8839555e admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.683413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b2bd1-b813-49fc-9788-c0e23c6e87fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.684096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b2bd1-b813-49fc-9788-c0e23c6e87fd admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.684412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b2bd1-b813-49fc-9788-c0e23c6e87fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.684668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b2bd1-b813-49fc-9788-c0e23c6e87fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.685416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b2bd1-b813-49fc-9788-c0e23c6e87fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:10.685539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8baab-66bf-44fb-984f-d6722558d9ff admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.685749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b2bd1-b813-49fc-9788-c0e23c6e87fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.687333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b2bd1-b813-49fc-9788-c0e23c6e87fd admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.690081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f6314-888a-4d22-a8bc-c9ddf505cebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.690741 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f6314-888a-4d22-a8bc-c9ddf505cebd admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.691023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f6314-888a-4d22-a8bc-c9ddf505cebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.691304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f6314-888a-4d22-a8bc-c9ddf505cebd admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.691996 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f6314-888a-4d22-a8bc-c9ddf505cebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.693925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f6314-888a-4d22-a8bc-c9ddf505cebd admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.699977 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 793/1568] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:30:10.700907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b2bd1-b813-49fc-9788-c0e23c6e87fd admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.710482 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b2bd1-b813-49fc-9788-c0e23c6e87fd admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.715406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa9aa9-fcb6-4de1-a5be-a92f2fe6694d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.716115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa9aa9-fcb6-4de1-a5be-a92f2fe6694d admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.716339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa9aa9-fcb6-4de1-a5be-a92f2fe6694d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.716608 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa9aa9-fcb6-4de1-a5be-a92f2fe6694d admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.717330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa9aa9-fcb6-4de1-a5be-a92f2fe6694d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.719240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa9aa9-fcb6-4de1-a5be-a92f2fe6694d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.724573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 776/1569] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:10.724573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f6314-888a-4d22-a8bc-c9ddf505cebd admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.742499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 794/1570] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/users => generated 289 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:10.742499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa9aa9-fcb6-4de1-a5be-a92f2fe6694d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.750525 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa9aa9-fcb6-4de1-a5be-a92f2fe6694d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.788980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f6314-888a-4d22-a8bc-c9ddf505cebd admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.794604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0e7e6-1031-4f79-9595-352c0285f7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.795268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0e7e6-1031-4f79-9595-352c0285f7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.795551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0e7e6-1031-4f79-9595-352c0285f7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.795828 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0e7e6-1031-4f79-9595-352c0285f7c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.796559 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0e7e6-1031-4f79-9595-352c0285f7c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.798178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0e7e6-1031-4f79-9595-352c0285f7c9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.809165 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 777/1571] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:10.809408 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776c382-e9bf-44fa-a7cf-54c3e800656a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.810156 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776c382-e9bf-44fa-a7cf-54c3e800656a admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.810438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0e7e6-1031-4f79-9595-352c0285f7c9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.810561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776c382-e9bf-44fa-a7cf-54c3e800656a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.810728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776c382-e9bf-44fa-a7cf-54c3e800656a admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.811436 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776c382-e9bf-44fa-a7cf-54c3e800656a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.813167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776c382-e9bf-44fa-a7cf-54c3e800656a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.821670 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0e7e6-1031-4f79-9595-352c0285f7c9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.823571 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 795/1572] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:10.824419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776c382-e9bf-44fa-a7cf-54c3e800656a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.829935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3358cea-4a7d-4db1-8387-cf5d306f881e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.830625 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3358cea-4a7d-4db1-8387-cf5d306f881e admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.830910 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3358cea-4a7d-4db1-8387-cf5d306f881e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.831188 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3358cea-4a7d-4db1-8387-cf5d306f881e admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.831882 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3358cea-4a7d-4db1-8387-cf5d306f881e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.833818 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3358cea-4a7d-4db1-8387-cf5d306f881e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.833944 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776c382-e9bf-44fa-a7cf-54c3e800656a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.839181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b812e0-f35e-489d-bb07-0ffd413e1bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.839789 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b812e0-f35e-489d-bb07-0ffd413e1bab admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.840074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b812e0-f35e-489d-bb07-0ffd413e1bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.840351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b812e0-f35e-489d-bb07-0ffd413e1bab admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.841034 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b812e0-f35e-489d-bb07-0ffd413e1bab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.843029 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b812e0-f35e-489d-bb07-0ffd413e1bab admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.861713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 778/1573] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:10.862567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3358cea-4a7d-4db1-8387-cf5d306f881e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.875323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 796/1574] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:10.875323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b812e0-f35e-489d-bb07-0ffd413e1bab admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.925068 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3358cea-4a7d-4db1-8387-cf5d306f881e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.934451 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:10.938917 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b812e0-f35e-489d-bb07-0ffd413e1bab admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:10.943911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725580b-7cbc-4282-827d-a98101db619b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.944581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725580b-7cbc-4282-827d-a98101db619b admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.944865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725580b-7cbc-4282-827d-a98101db619b admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.945157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725580b-7cbc-4282-827d-a98101db619b admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.945872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725580b-7cbc-4282-827d-a98101db619b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.947828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725580b-7cbc-4282-827d-a98101db619b admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:10.961633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:10.975653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 797/1575] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:10.976449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725580b-7cbc-4282-827d-a98101db619b admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:10.987014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701c036-0557-4dd1-99c7-438f9c63643b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:10.987683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701c036-0557-4dd1-99c7-438f9c63643b admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:10.988088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701c036-0557-4dd1-99c7-438f9c63643b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:10.988372 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701c036-0557-4dd1-99c7-438f9c63643b admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:10.989181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701c036-0557-4dd1-99c7-438f9c63643b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:10.991100 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701c036-0557-4dd1-99c7-438f9c63643b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.018576 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 779/1576] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/projects => generated 403 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:11.022157 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701c036-0557-4dd1-99c7-438f9c63643b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.039209 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725580b-7cbc-4282-827d-a98101db619b admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.084565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701c036-0557-4dd1-99c7-438f9c63643b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.089985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c4e37-38b7-4491-95ae-09d8efa2c423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.090126 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c4e37-38b7-4491-95ae-09d8efa2c423 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.090332 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c4e37-38b7-4491-95ae-09d8efa2c423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.090611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c4e37-38b7-4491-95ae-09d8efa2c423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.091375 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c4e37-38b7-4491-95ae-09d8efa2c423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.092985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c4e37-38b7-4491-95ae-09d8efa2c423 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.095933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54553eb8-4a8b-46d6-8210-5fbc440c837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.096611 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54553eb8-4a8b-46d6-8210-5fbc440c837f admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.096902 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54553eb8-4a8b-46d6-8210-5fbc440c837f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.097206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54553eb8-4a8b-46d6-8210-5fbc440c837f admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.097903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54553eb8-4a8b-46d6-8210-5fbc440c837f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.099827 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54553eb8-4a8b-46d6-8210-5fbc440c837f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.103376 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 780/1577] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:11.104822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c4e37-38b7-4491-95ae-09d8efa2c423 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.114450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c4e37-38b7-4491-95ae-09d8efa2c423 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.120021 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723639b-5648-4132-8f75-13f3a3613b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.120729 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723639b-5648-4132-8f75-13f3a3613b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.121012 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723639b-5648-4132-8f75-13f3a3613b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.121320 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723639b-5648-4132-8f75-13f3a3613b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.124595 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 798/1578] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:10 2022] POST /identity/v3/users => generated 294 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:11.124745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723639b-5648-4132-8f75-13f3a3613b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.124849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54553eb8-4a8b-46d6-8210-5fbc440c837f admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.124947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723639b-5648-4132-8f75-13f3a3613b64 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.137293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 781/1579] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:11.137477 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54553eb8-4a8b-46d6-8210-5fbc440c837f admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.137594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723639b-5648-4132-8f75-13f3a3613b64 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.141532 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b95476-ab3a-4bf5-bb10-f1e44c6f9f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.142195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b95476-ab3a-4bf5-bb10-f1e44c6f9f45 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.142481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b95476-ab3a-4bf5-bb10-f1e44c6f9f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.142761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b95476-ab3a-4bf5-bb10-f1e44c6f9f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.143498 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b95476-ab3a-4bf5-bb10-f1e44c6f9f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.145106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b95476-ab3a-4bf5-bb10-f1e44c6f9f45 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.146796 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723639b-5648-4132-8f75-13f3a3613b64 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.151852 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055ce99-6438-47aa-ae2b-0ec6473da3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.152482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055ce99-6438-47aa-ae2b-0ec6473da3ed admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.152765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055ce99-6438-47aa-ae2b-0ec6473da3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.153046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055ce99-6438-47aa-ae2b-0ec6473da3ed admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.153798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055ce99-6438-47aa-ae2b-0ec6473da3ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.155682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055ce99-6438-47aa-ae2b-0ec6473da3ed admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.156501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 799/1580] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:11.157316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b95476-ab3a-4bf5-bb10-f1e44c6f9f45 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.169049 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b95476-ab3a-4bf5-bb10-f1e44c6f9f45 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.174866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20037e4-f5aa-4a4e-b152-7e6a9dab9d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.175540 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20037e4-f5aa-4a4e-b152-7e6a9dab9d66 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.175957 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20037e4-f5aa-4a4e-b152-7e6a9dab9d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.176239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20037e4-f5aa-4a4e-b152-7e6a9dab9d66 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.176939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20037e4-f5aa-4a4e-b152-7e6a9dab9d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.178258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 782/1581] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:11 2022] POST /identity/v3/users => generated 286 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:11.178924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20037e4-f5aa-4a4e-b152-7e6a9dab9d66 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.179044 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055ce99-6438-47aa-ae2b-0ec6473da3ed admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.191246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055ce99-6438-47aa-ae2b-0ec6473da3ed admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.196138 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754bc94-60d6-4ed1-b614-8fa827dcf9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.196804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754bc94-60d6-4ed1-b614-8fa827dcf9c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.197088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754bc94-60d6-4ed1-b614-8fa827dcf9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.197469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754bc94-60d6-4ed1-b614-8fa827dcf9c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.198113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754bc94-60d6-4ed1-b614-8fa827dcf9c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.200039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754bc94-60d6-4ed1-b614-8fa827dcf9c7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.200256 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 800/1582] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:11 2022] POST /identity/v3/users => generated 291 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:11.201205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20037e4-f5aa-4a4e-b152-7e6a9dab9d66 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.210851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20037e4-f5aa-4a4e-b152-7e6a9dab9d66 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.215914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd1050-bfa1-42b9-850d-8aa56decd3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.216574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd1050-bfa1-42b9-850d-8aa56decd3aa admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.216879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd1050-bfa1-42b9-850d-8aa56decd3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.217129 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd1050-bfa1-42b9-850d-8aa56decd3aa admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.217838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd1050-bfa1-42b9-850d-8aa56decd3aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.223202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd1050-bfa1-42b9-850d-8aa56decd3aa admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.223353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 783/1583] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:11 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:11.223353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754bc94-60d6-4ed1-b614-8fa827dcf9c7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.233517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754bc94-60d6-4ed1-b614-8fa827dcf9c7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.238000 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cda93a-a88c-40f8-9d0a-2ec7f2d41b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.238659 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cda93a-a88c-40f8-9d0a-2ec7f2d41b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.239021 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cda93a-a88c-40f8-9d0a-2ec7f2d41b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.239302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cda93a-a88c-40f8-9d0a-2ec7f2d41b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.242333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cda93a-a88c-40f8-9d0a-2ec7f2d41b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.242333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cda93a-a88c-40f8-9d0a-2ec7f2d41b43 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.249573 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 801/1584] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:11 2022] POST /identity/v3/users => generated 302 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:11.250434 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd1050-bfa1-42b9-850d-8aa56decd3aa admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.256057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 784/1585] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:11.256871 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cda93a-a88c-40f8-9d0a-2ec7f2d41b43 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.263040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd1050-bfa1-42b9-850d-8aa56decd3aa admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.266620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cda93a-a88c-40f8-9d0a-2ec7f2d41b43 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.267752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b84772-21e9-43ab-8efe-16b1d839b973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.268414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b84772-21e9-43ab-8efe-16b1d839b973 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.268695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b84772-21e9-43ab-8efe-16b1d839b973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.268967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b84772-21e9-43ab-8efe-16b1d839b973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.269847 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b84772-21e9-43ab-8efe-16b1d839b973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.271335 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1aac0-b46e-4b8c-b3c8-c27f9278ae8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.271470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b84772-21e9-43ab-8efe-16b1d839b973 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.272200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1aac0-b46e-4b8c-b3c8-c27f9278ae8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.272488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1aac0-b46e-4b8c-b3c8-c27f9278ae8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.272762 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1aac0-b46e-4b8c-b3c8-c27f9278ae8b admin admin] PATH_INFO: `/v3/projects/2deb7182f86846e09269b7cba6fab758/users/064d7bbcf3684ccbafc5f3d1cb561d9a/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.282998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 802/1586] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:11.283878 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b84772-21e9-43ab-8efe-16b1d839b973 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.294230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b84772-21e9-43ab-8efe-16b1d839b973 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.294648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1aac0-b46e-4b8c-b3c8-c27f9278ae8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2deb7182f86846e09269b7cba6fab758, user_id=064d7bbcf3684ccbafc5f3d1cb561d9a, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.295835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-6ea1aac0-b46e-4b8c-b3c8-c27f9278ae8b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 15:30:11.295835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) Jul 19 15:30:11.295835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: Jul 19 15:30:11.298009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1aac0-b46e-4b8c-b3c8-c27f9278ae8b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.298795 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbd4fa-adf3-4a18-b437-44788ea25d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.299665 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbd4fa-adf3-4a18-b437-44788ea25d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.299935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbd4fa-adf3-4a18-b437-44788ea25d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.300225 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbd4fa-adf3-4a18-b437-44788ea25d4f admin admin] PATH_INFO: `/v3/projects/2e61067f09b04588ac46e5f74a29a22f/users/3b7365e66adc407cbe719e5bd5558008/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.311454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 785/1587] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:11 2022] PUT /identity/v3/projects/2deb7182f86846e09269b7cba6fab758/users/064d7bbcf3684ccbafc5f3d1cb561d9a/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:11.312335 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1aac0-b46e-4b8c-b3c8-c27f9278ae8b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.316860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbd4fa-adf3-4a18-b437-44788ea25d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e61067f09b04588ac46e5f74a29a22f, user_id=3b7365e66adc407cbe719e5bd5558008, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.319629 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbd4fa-adf3-4a18-b437-44788ea25d4f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.326490 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:11.332662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 803/1588] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:11 2022] PUT /identity/v3/projects/2e61067f09b04588ac46e5f74a29a22f/users/3b7365e66adc407cbe719e5bd5558008/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:11.333444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbd4fa-adf3-4a18-b437-44788ea25d4f admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.352806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:11.381682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1aac0-b46e-4b8c-b3c8-c27f9278ae8b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.413915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbd4fa-adf3-4a18-b437-44788ea25d4f admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.419426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9630e68-9a58-4279-b7a8-d73b1334607a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.420092 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9630e68-9a58-4279-b7a8-d73b1334607a admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.420376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9630e68-9a58-4279-b7a8-d73b1334607a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.420651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9630e68-9a58-4279-b7a8-d73b1334607a admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.421414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9630e68-9a58-4279-b7a8-d73b1334607a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.423023 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9630e68-9a58-4279-b7a8-d73b1334607a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.435889 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 804/1589] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:11.435889 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9630e68-9a58-4279-b7a8-d73b1334607a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.440430 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2bf46-4f37-48df-bd72-fef310ddae4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.441106 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2bf46-4f37-48df-bd72-fef310ddae4a admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.441418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2bf46-4f37-48df-bd72-fef310ddae4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.441697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2bf46-4f37-48df-bd72-fef310ddae4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.442432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2bf46-4f37-48df-bd72-fef310ddae4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.442866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9630e68-9a58-4279-b7a8-d73b1334607a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.444058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2bf46-4f37-48df-bd72-fef310ddae4a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.447414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df46cd1-0e11-4afb-aef6-84dfcb75306b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.448322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df46cd1-0e11-4afb-aef6-84dfcb75306b admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.448679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df46cd1-0e11-4afb-aef6-84dfcb75306b admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.448957 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df46cd1-0e11-4afb-aef6-84dfcb75306b admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.449708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df46cd1-0e11-4afb-aef6-84dfcb75306b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.451257 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df46cd1-0e11-4afb-aef6-84dfcb75306b admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.454151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 786/1590] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:11.454770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2bf46-4f37-48df-bd72-fef310ddae4a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.466837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 805/1591] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:11.466837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df46cd1-0e11-4afb-aef6-84dfcb75306b admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.467134 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2bf46-4f37-48df-bd72-fef310ddae4a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.470655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53daa454-9bfb-4d6e-9129-f097eddef3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.471308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53daa454-9bfb-4d6e-9129-f097eddef3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.471593 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53daa454-9bfb-4d6e-9129-f097eddef3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.471873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53daa454-9bfb-4d6e-9129-f097eddef3a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.472599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53daa454-9bfb-4d6e-9129-f097eddef3a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.474222 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53daa454-9bfb-4d6e-9129-f097eddef3a9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.477303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df46cd1-0e11-4afb-aef6-84dfcb75306b admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.483765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3288f-1411-497f-aa52-d8da064e9142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.484576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3288f-1411-497f-aa52-d8da064e9142 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.484858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3288f-1411-497f-aa52-d8da064e9142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.485269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3288f-1411-497f-aa52-d8da064e9142 admin admin] PATH_INFO: `/v3/projects/7d55d02d809143f682698fbb494f65f4/users/51eb5efc37ca4942831bc4d046f54da6/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.485388 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 787/1592] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:11.485979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53daa454-9bfb-4d6e-9129-f097eddef3a9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.495334 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53daa454-9bfb-4d6e-9129-f097eddef3a9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.499944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc89136-800c-4ec5-bdf8-e49c3eb827d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.500078 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3288f-1411-497f-aa52-d8da064e9142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d55d02d809143f682698fbb494f65f4, user_id=51eb5efc37ca4942831bc4d046f54da6, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.500585 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc89136-800c-4ec5-bdf8-e49c3eb827d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.500869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc89136-800c-4ec5-bdf8-e49c3eb827d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.501168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc89136-800c-4ec5-bdf8-e49c3eb827d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.501892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc89136-800c-4ec5-bdf8-e49c3eb827d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.502735 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3288f-1411-497f-aa52-d8da064e9142 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.503454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc89136-800c-4ec5-bdf8-e49c3eb827d2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.514657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 788/1593] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:11.515835 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 806/1594] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:11 2022] PUT /identity/v3/projects/7d55d02d809143f682698fbb494f65f4/users/51eb5efc37ca4942831bc4d046f54da6/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:11.516248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4394d-7983-434d-8f50-e0f3c1e89a89 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.516637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4394d-7983-434d-8f50-e0f3c1e89a89 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.516915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdac508-f235-4ce5-b5d0-352e56c8db91 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.517102 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4394d-7983-434d-8f50-e0f3c1e89a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.517250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdac508-f235-4ce5-b5d0-352e56c8db91 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.517507 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdac508-f235-4ce5-b5d0-352e56c8db91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.536316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-59f4394d-7983-434d-8f50-e0f3c1e89a89 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:11.537713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-59f4394d-7983-434d-8f50-e0f3c1e89a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:11.545248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-cfdac508-f235-4ce5-b5d0-352e56c8db91 None None] MFA Rules not processed for user `3b7365e66adc407cbe719e5bd5558008`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:11.546487 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cfdac508-f235-4ce5-b5d0-352e56c8db91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:11.601035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 789/1595] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:11.601829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4394d-7983-434d-8f50-e0f3c1e89a89 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.612307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4394d-7983-434d-8f50-e0f3c1e89a89 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.619309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 807/1596] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:11.619309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdac508-f235-4ce5-b5d0-352e56c8db91 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.621473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d9c88-ca0c-4cf2-b9c0-d12fcd967f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.622152 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d9c88-ca0c-4cf2-b9c0-d12fcd967f05 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.622439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d9c88-ca0c-4cf2-b9c0-d12fcd967f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.622722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d9c88-ca0c-4cf2-b9c0-d12fcd967f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.623458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d9c88-ca0c-4cf2-b9c0-d12fcd967f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.625248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d9c88-ca0c-4cf2-b9c0-d12fcd967f05 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.628097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdac508-f235-4ce5-b5d0-352e56c8db91 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.632588 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5f80a-99a3-4e2a-be3e-18e825148c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.633262 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5f80a-99a3-4e2a-be3e-18e825148c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.633543 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5f80a-99a3-4e2a-be3e-18e825148c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.633814 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5f80a-99a3-4e2a-be3e-18e825148c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.634537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e5f80a-99a3-4e2a-be3e-18e825148c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.636093 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 790/1597] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:11.636799 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d9c88-ca0c-4cf2-b9c0-d12fcd967f05 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.636920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e5f80a-99a3-4e2a-be3e-18e825148c85 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.646558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d9c88-ca0c-4cf2-b9c0-d12fcd967f05 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.648306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 808/1598] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:11.649101 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5f80a-99a3-4e2a-be3e-18e825148c85 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.652467 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcc139-84e8-4c91-9497-1a479c718db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.653122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcc139-84e8-4c91-9497-1a479c718db3 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.653432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcc139-84e8-4c91-9497-1a479c718db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.653710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcc139-84e8-4c91-9497-1a479c718db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.654427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcc139-84e8-4c91-9497-1a479c718db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.656003 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcc139-84e8-4c91-9497-1a479c718db3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.659521 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5f80a-99a3-4e2a-be3e-18e825148c85 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.664142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca964923-0b8c-4561-9a2e-6924f6d5a3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.664784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca964923-0b8c-4561-9a2e-6924f6d5a3a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.665076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca964923-0b8c-4561-9a2e-6924f6d5a3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.665389 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca964923-0b8c-4561-9a2e-6924f6d5a3a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.666116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca964923-0b8c-4561-9a2e-6924f6d5a3a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.666265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 791/1599] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:11.667330 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcc139-84e8-4c91-9497-1a479c718db3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.667785 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca964923-0b8c-4561-9a2e-6924f6d5a3a6 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.678451 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcc139-84e8-4c91-9497-1a479c718db3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.678617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 809/1600] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:11.679696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99db68-340e-4eae-8ebe-637c8f4e41da None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.679994 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99db68-340e-4eae-8ebe-637c8f4e41da None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.680277 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99db68-340e-4eae-8ebe-637c8f4e41da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.684046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458e312-d558-4fc8-9056-f2d78928eefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.684926 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458e312-d558-4fc8-9056-f2d78928eefb admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.685239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458e312-d558-4fc8-9056-f2d78928eefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.685500 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458e312-d558-4fc8-9056-f2d78928eefb admin admin] PATH_INFO: `/v3/projects/9bf94c44535545689ddc74bc82888a46/users/ddb5e1dec1904ae183e131ba95ed1ef2/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.696817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:11.699312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-0f99db68-340e-4eae-8ebe-637c8f4e41da None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:11.700753 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0f99db68-340e-4eae-8ebe-637c8f4e41da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:11.702342 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458e312-d558-4fc8-9056-f2d78928eefb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf94c44535545689ddc74bc82888a46, user_id=ddb5e1dec1904ae183e131ba95ed1ef2, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.705003 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458e312-d558-4fc8-9056-f2d78928eefb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.708523 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 810/1601] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:11.709401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99db68-340e-4eae-8ebe-637c8f4e41da None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.710873 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0f99db68-340e-4eae-8ebe-637c8f4e41da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:11.721802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 792/1602] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:11 2022] PUT /identity/v3/projects/9bf94c44535545689ddc74bc82888a46/users/ddb5e1dec1904ae183e131ba95ed1ef2/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:11.722782 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458e312-d558-4fc8-9056-f2d78928eefb admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.749377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:11.773729 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99db68-340e-4eae-8ebe-637c8f4e41da None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.786583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458e312-d558-4fc8-9056-f2d78928eefb admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.791343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281ab68-f84e-499f-b5ac-0ef1480bb440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.792144 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281ab68-f84e-499f-b5ac-0ef1480bb440 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.792435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281ab68-f84e-499f-b5ac-0ef1480bb440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.792719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281ab68-f84e-499f-b5ac-0ef1480bb440 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.793452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281ab68-f84e-499f-b5ac-0ef1480bb440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.795374 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281ab68-f84e-499f-b5ac-0ef1480bb440 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.824758 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e22083-51d0-4100-88a8-bc88f4384541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.825451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e22083-51d0-4100-88a8-bc88f4384541 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.825732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e22083-51d0-4100-88a8-bc88f4384541 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.826026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e22083-51d0-4100-88a8-bc88f4384541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.827544 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 793/1603] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:11 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:11.828602 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281ab68-f84e-499f-b5ac-0ef1480bb440 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.838184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e22083-51d0-4100-88a8-bc88f4384541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.839669 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e22083-51d0-4100-88a8-bc88f4384541 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.849268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 811/1604] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:11.850039 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e22083-51d0-4100-88a8-bc88f4384541 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.859505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e22083-51d0-4100-88a8-bc88f4384541 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.898855 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281ab68-f84e-499f-b5ac-0ef1480bb440 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:11.904042 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99daf4-d598-4047-ad81-8c3da65d3e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.904983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99daf4-d598-4047-ad81-8c3da65d3e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.905250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99daf4-d598-4047-ad81-8c3da65d3e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.905529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99daf4-d598-4047-ad81-8c3da65d3e31 admin admin] PATH_INFO: `/v3/projects/8c457bf7357846e59a4db9c3492463a5/users/2123408c9dc0442b83a36e95efa04ee8/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.922504 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99daf4-d598-4047-ad81-8c3da65d3e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c457bf7357846e59a4db9c3492463a5, user_id=2123408c9dc0442b83a36e95efa04ee8, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.925203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99daf4-d598-4047-ad81-8c3da65d3e31 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.929458 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6641a1f-9697-4018-8e8c-32b9f227ae8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:11.930516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6641a1f-9697-4018-8e8c-32b9f227ae8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:11.930907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6641a1f-9697-4018-8e8c-32b9f227ae8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:11.931230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6641a1f-9697-4018-8e8c-32b9f227ae8d admin admin] PATH_INFO: `/v3/projects/732ecfbf08874525a7ecbdec9abf34eb/users/b9dfaf168adf45618c828806ec285026/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:11.948838 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 794/1605] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:11 2022] PUT /identity/v3/projects/8c457bf7357846e59a4db9c3492463a5/users/2123408c9dc0442b83a36e95efa04ee8/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:11.948838 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99daf4-d598-4047-ad81-8c3da65d3e31 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:11.956204 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6641a1f-9697-4018-8e8c-32b9f227ae8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=732ecfbf08874525a7ecbdec9abf34eb, user_id=b9dfaf168adf45618c828806ec285026, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:11.959156 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6641a1f-9697-4018-8e8c-32b9f227ae8d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:11.972237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 812/1606] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:11 2022] PUT /identity/v3/projects/732ecfbf08874525a7ecbdec9abf34eb/users/b9dfaf168adf45618c828806ec285026/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:11.973378 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6641a1f-9697-4018-8e8c-32b9f227ae8d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:12.019670 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:12.023524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99daf4-d598-4047-ad81-8c3da65d3e31 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:12.039838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6641a1f-9697-4018-8e8c-32b9f227ae8d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:12.045311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03408c24-8f73-43c4-940e-067ad3dcdc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:12.046237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03408c24-8f73-43c4-940e-067ad3dcdc9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.046597 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03408c24-8f73-43c4-940e-067ad3dcdc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.046992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03408c24-8f73-43c4-940e-067ad3dcdc9f admin admin] PATH_INFO: `/v3/projects/3eac7356ebf340139bbc28015ec21472/users/46cd0aa86f274786b53c492a72c199cd/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.063277 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03408c24-8f73-43c4-940e-067ad3dcdc9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eac7356ebf340139bbc28015ec21472, user_id=46cd0aa86f274786b53c492a72c199cd, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:12.066076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03408c24-8f73-43c4-940e-067ad3dcdc9f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:12.078488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:12.085969 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 813/1607] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:11 2022] PUT /identity/v3/projects/3eac7356ebf340139bbc28015ec21472/users/46cd0aa86f274786b53c492a72c199cd/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:12.091252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03408c24-8f73-43c4-940e-067ad3dcdc9f admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:12.116665 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b593d40-8e78-4920-8fff-6cbd2de3d1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:12.117707 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b593d40-8e78-4920-8fff-6cbd2de3d1bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.118005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b593d40-8e78-4920-8fff-6cbd2de3d1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.118285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b593d40-8e78-4920-8fff-6cbd2de3d1bb admin admin] PATH_INFO: `/v3/projects/af417e926a2d4d328a37fbb7baedbe78/users/5507329a6bb84cfbbdac99b85f346ab8/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.135037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b593d40-8e78-4920-8fff-6cbd2de3d1bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=af417e926a2d4d328a37fbb7baedbe78, user_id=5507329a6bb84cfbbdac99b85f346ab8, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:12.138235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b593d40-8e78-4920-8fff-6cbd2de3d1bb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:12.150174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03408c24-8f73-43c4-940e-067ad3dcdc9f admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:12.154387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 795/1608] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:11 2022] PUT /identity/v3/projects/af417e926a2d4d328a37fbb7baedbe78/users/5507329a6bb84cfbbdac99b85f346ab8/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:12.155705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7568ece-2614-42c0-96f5-5ab15c783ce4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.155995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7568ece-2614-42c0-96f5-5ab15c783ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.156277 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7568ece-2614-42c0-96f5-5ab15c783ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.174991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a7568ece-2614-42c0-96f5-5ab15c783ce4 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:12.176329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a7568ece-2614-42c0-96f5-5ab15c783ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:12.211740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a91a0-f0d6-4938-81a0-0dff4d3f3c72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:12.212497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a91a0-f0d6-4938-81a0-0dff4d3f3c72 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.212842 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a91a0-f0d6-4938-81a0-0dff4d3f3c72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.213253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a91a0-f0d6-4938-81a0-0dff4d3f3c72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.262021 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 796/1609] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:12.263214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7568ece-2614-42c0-96f5-5ab15c783ce4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:12.276086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7568ece-2614-42c0-96f5-5ab15c783ce4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:12.283015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df0529-6f1e-48ca-9e8f-e9c57615f343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:12.284038 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df0529-6f1e-48ca-9e8f-e9c57615f343 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.284426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df0529-6f1e-48ca-9e8f-e9c57615f343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.284975 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df0529-6f1e-48ca-9e8f-e9c57615f343 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.286048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df0529-6f1e-48ca-9e8f-e9c57615f343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:12.289263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df0529-6f1e-48ca-9e8f-e9c57615f343 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:12.310135 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a91a0-f0d6-4938-81a0-0dff4d3f3c72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:12.311793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a91a0-f0d6-4938-81a0-0dff4d3f3c72 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:12.317063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 797/1610] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:12 2022] POST /identity/v3/users => generated 300 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:12.318662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e0d6f-7fab-4ae7-bf28-fc41a08aca72 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.319156 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e0d6f-7fab-4ae7-bf28-fc41a08aca72 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.320151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e0d6f-7fab-4ae7-bf28-fc41a08aca72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.329813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 814/1611] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:12.333992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cc7bd-ee26-41cd-a248-728f88b1f236 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.334587 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cc7bd-ee26-41cd-a248-728f88b1f236 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.335022 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cc7bd-ee26-41cd-a248-728f88b1f236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.362140 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a68e0d6f-7fab-4ae7-bf28-fc41a08aca72 None None] MFA Rules not processed for user `2123408c9dc0442b83a36e95efa04ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:12.364055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a68e0d6f-7fab-4ae7-bf28-fc41a08aca72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:12.376539 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-858cc7bd-ee26-41cd-a248-728f88b1f236 None None] MFA Rules not processed for user `b9dfaf168adf45618c828806ec285026`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:12.378169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-858cc7bd-ee26-41cd-a248-728f88b1f236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:12.450709 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:12.456526 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:12.463845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 815/1612] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:12.465166 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 798/1613] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:12.465581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb42bcf-47dd-4ac9-990f-1d75ab7e0a17 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.466158 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb42bcf-47dd-4ac9-990f-1d75ab7e0a17 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.466266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb42bcf-47dd-4ac9-990f-1d75ab7e0a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.470685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df1cc1-b106-4f7c-9eee-e059b4ade45b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.471179 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df1cc1-b106-4f7c-9eee-e059b4ade45b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.471671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df1cc1-b106-4f7c-9eee-e059b4ade45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.498087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d2df1cc1-b106-4f7c-9eee-e059b4ade45b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:12.499811 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d2df1cc1-b106-4f7c-9eee-e059b4ade45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:12.515528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 799/1614] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:12.515528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df1cc1-b106-4f7c-9eee-e059b4ade45b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:12.521741 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ecb42bcf-47dd-4ac9-990f-1d75ab7e0a17 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:12.525408 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ecb42bcf-47dd-4ac9-990f-1d75ab7e0a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:12.529376 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df1cc1-b106-4f7c-9eee-e059b4ade45b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:12.538350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf51cd02-4262-4f4a-b170-e18eafe276d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:12.539105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51cd02-4262-4f4a-b170-e18eafe276d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.540324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51cd02-4262-4f4a-b170-e18eafe276d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.540616 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51cd02-4262-4f4a-b170-e18eafe276d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.547305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 816/1615] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:12.547305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb42bcf-47dd-4ac9-990f-1d75ab7e0a17 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:12.555900 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb42bcf-47dd-4ac9-990f-1d75ab7e0a17 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:12.560360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf51cd02-4262-4f4a-b170-e18eafe276d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:12.561964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612d9b8-f29b-40e2-86a1-a090a604ac4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:12.562099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf51cd02-4262-4f4a-b170-e18eafe276d1 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:12.562920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612d9b8-f29b-40e2-86a1-a090a604ac4d admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.563213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612d9b8-f29b-40e2-86a1-a090a604ac4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.563496 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612d9b8-f29b-40e2-86a1-a090a604ac4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.564395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612d9b8-f29b-40e2-86a1-a090a604ac4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:12.566727 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612d9b8-f29b-40e2-86a1-a090a604ac4d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:12.577839 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 800/1616] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:12.578757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf51cd02-4262-4f4a-b170-e18eafe276d1 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:12.584005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 817/1617] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:12.584005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83ea58-830a-45a5-8ca1-b5cbe7ec04e0 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.584005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83ea58-830a-45a5-8ca1-b5cbe7ec04e0 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.584005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83ea58-830a-45a5-8ca1-b5cbe7ec04e0 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.585593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 818/1618] 10.222.0.42 () {54 vars in 962 bytes} [Tue Jul 19 15:30:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:30:12.586427 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83ea58-830a-45a5-8ca1-b5cbe7ec04e0 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:12.598620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf51cd02-4262-4f4a-b170-e18eafe276d1 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:12.603014 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83ea58-830a-45a5-8ca1-b5cbe7ec04e0 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:12.606469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a2569-c553-484f-a997-737e7d579349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:12.607288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a2569-c553-484f-a997-737e7d579349 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.607529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a2569-c553-484f-a997-737e7d579349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.607841 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a2569-c553-484f-a997-737e7d579349 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.608910 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a2569-c553-484f-a997-737e7d579349 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:12.609793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58b4b3-e183-4cc0-a8b1-4b7fe635ae3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:12.611262 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58b4b3-e183-4cc0-a8b1-4b7fe635ae3c service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.611388 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a2569-c553-484f-a997-737e7d579349 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:12.611489 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58b4b3-e183-4cc0-a8b1-4b7fe635ae3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.611489 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58b4b3-e183-4cc0-a8b1-4b7fe635ae3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.630027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58b4b3-e183-4cc0-a8b1-4b7fe635ae3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:12.633158 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58b4b3-e183-4cc0-a8b1-4b7fe635ae3c service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:12.654232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 819/1619] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:12.654450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 801/1620] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:12 2022] POST /identity/v3/projects => generated 381 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:12.655542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58b4b3-e183-4cc0-a8b1-4b7fe635ae3c service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:12.660303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a2569-c553-484f-a997-737e7d579349 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:12.669298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58b4b3-e183-4cc0-a8b1-4b7fe635ae3c service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:12.751877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a2569-c553-484f-a997-737e7d579349 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:12.756037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7fd9d-ae45-4126-842a-1f1e95406ef3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:12.756620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7fd9d-ae45-4126-842a-1f1e95406ef3 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.756903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7fd9d-ae45-4126-842a-1f1e95406ef3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.757210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7fd9d-ae45-4126-842a-1f1e95406ef3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.794996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46199dcb-4066-479e-86d0-6e04e0eba405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:12.795845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46199dcb-4066-479e-86d0-6e04e0eba405 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.796121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46199dcb-4066-479e-86d0-6e04e0eba405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.796682 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46199dcb-4066-479e-86d0-6e04e0eba405 admin admin] PATH_INFO: `/v3/projects/58a8a875a16c46ecb7a0d4d845183ba4/users/97668cce1d0e45ba936f651393e7a794/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.831681 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7fd9d-ae45-4126-842a-1f1e95406ef3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:12.833024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7fd9d-ae45-4126-842a-1f1e95406ef3 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:12.835537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46199dcb-4066-479e-86d0-6e04e0eba405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a8a875a16c46ecb7a0d4d845183ba4, user_id=97668cce1d0e45ba936f651393e7a794, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:12.838314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46199dcb-4066-479e-86d0-6e04e0eba405 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:12.845373 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 802/1621] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:12.846963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13e147-d00f-4329-9dda-868bb99ee6d0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:12.847187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13e147-d00f-4329-9dda-868bb99ee6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:12.847461 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13e147-d00f-4329-9dda-868bb99ee6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:12.868941 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 820/1622] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:12 2022] PUT /identity/v3/projects/58a8a875a16c46ecb7a0d4d845183ba4/users/97668cce1d0e45ba936f651393e7a794/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:12.870515 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46199dcb-4066-479e-86d0-6e04e0eba405 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:12.898935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-fe13e147-d00f-4329-9dda-868bb99ee6d0 None None] MFA Rules not processed for user `bff6a2d4bc734904b2615608a86b097d`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:12.900481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fe13e147-d00f-4329-9dda-868bb99ee6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:12.955203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:12.988979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 803/1623] 10.222.0.42 () {58 vars in 1059 bytes} [Tue Jul 19 15:30:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:12.990600 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13e147-d00f-4329-9dda-868bb99ee6d0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:13.035326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46199dcb-4066-479e-86d0-6e04e0eba405 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:13.049501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea87be-7313-4024-9750-cb2b6aeb622b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:13.050579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea87be-7313-4024-9750-cb2b6aeb622b admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:13.050966 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea87be-7313-4024-9750-cb2b6aeb622b admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:13.051475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea87be-7313-4024-9750-cb2b6aeb622b admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:13.052450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea87be-7313-4024-9750-cb2b6aeb622b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:13.055044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea87be-7313-4024-9750-cb2b6aeb622b admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:13.074318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13e147-d00f-4329-9dda-868bb99ee6d0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:13.088356 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94727b-7275-4279-9fb3-a129c8506910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:13.089262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94727b-7275-4279-9fb3-a129c8506910 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:13.089547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94727b-7275-4279-9fb3-a129c8506910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:13.089826 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94727b-7275-4279-9fb3-a129c8506910 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:13.091026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df94727b-7275-4279-9fb3-a129c8506910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:13.092170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df94727b-7275-4279-9fb3-a129c8506910 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:13.116076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 821/1624] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:12 2022] POST /identity/v3/users => generated 297 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:13.117085 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea87be-7313-4024-9750-cb2b6aeb622b admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:13.122387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 804/1625] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:13.133805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94727b-7275-4279-9fb3-a129c8506910 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:13.140318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea87be-7313-4024-9750-cb2b6aeb622b admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:13.145302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94727b-7275-4279-9fb3-a129c8506910 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:13.148388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c91f2b-f97c-4693-91df-01bb8a94fcb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:13.150447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c91f2b-f97c-4693-91df-01bb8a94fcb8 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:13.150447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c91f2b-f97c-4693-91df-01bb8a94fcb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:13.151169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c91f2b-f97c-4693-91df-01bb8a94fcb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:13.164259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad847be-fbca-49fc-9250-20acd9dc60bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:13.165178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad847be-fbca-49fc-9250-20acd9dc60bf admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:13.165499 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad847be-fbca-49fc-9250-20acd9dc60bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:13.165779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad847be-fbca-49fc-9250-20acd9dc60bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:13.167443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad847be-fbca-49fc-9250-20acd9dc60bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:13.170376 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad847be-fbca-49fc-9250-20acd9dc60bf admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:13.190742 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 805/1626] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:13.191255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad847be-fbca-49fc-9250-20acd9dc60bf admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:13.203631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad847be-fbca-49fc-9250-20acd9dc60bf admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:13.211217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b8ed7-86c0-4cb1-abff-db603bfc491f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:13.212191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:13.212516 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b8ed7-86c0-4cb1-abff-db603bfc491f admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:13.212813 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b8ed7-86c0-4cb1-abff-db603bfc491f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:13.213475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b8ed7-86c0-4cb1-abff-db603bfc491f admin admin] PATH_INFO: `/v3/projects/58a8a875a16c46ecb7a0d4d845183ba4/users/97668cce1d0e45ba936f651393e7a794/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:13.216360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b8ed7-86c0-4cb1-abff-db603bfc491f admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a8a875a16c46ecb7a0d4d845183ba4, user_id=97668cce1d0e45ba936f651393e7a794, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:13.219593 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b8ed7-86c0-4cb1-abff-db603bfc491f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:13.238019 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 806/1627] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:13 2022] PUT /identity/v3/projects/58a8a875a16c46ecb7a0d4d845183ba4/users/97668cce1d0e45ba936f651393e7a794/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:13.248162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b8ed7-86c0-4cb1-abff-db603bfc491f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:13.256589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c91f2b-f97c-4693-91df-01bb8a94fcb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:13.258459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c91f2b-f97c-4693-91df-01bb8a94fcb8 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:13.272327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 822/1628] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:13.274258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f338cd-cba4-4490-8919-0f226b1894b2 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:13.275147 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f338cd-cba4-4490-8919-0f226b1894b2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:13.275653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f338cd-cba4-4490-8919-0f226b1894b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:13.318243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-24f338cd-cba4-4490-8919-0f226b1894b2 None None] MFA Rules not processed for user `97668cce1d0e45ba936f651393e7a794`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:13.320011 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-24f338cd-cba4-4490-8919-0f226b1894b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:13.414704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b8ed7-86c0-4cb1-abff-db603bfc491f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:13.419618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d594c0-17d9-4797-a48d-28ad7bd19354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:13.420524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d594c0-17d9-4797-a48d-28ad7bd19354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:13.420956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d594c0-17d9-4797-a48d-28ad7bd19354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:13.421436 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d594c0-17d9-4797-a48d-28ad7bd19354 admin admin] PATH_INFO: `/v3/projects/52837239ebfd465ebaccc583f4f1b1ac/users/20166c4e5f34486898274460d1f5bbaf/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:13.438935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d594c0-17d9-4797-a48d-28ad7bd19354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52837239ebfd465ebaccc583f4f1b1ac, user_id=20166c4e5f34486898274460d1f5bbaf, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:13.442128 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d594c0-17d9-4797-a48d-28ad7bd19354 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:13.469385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 807/1629] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:13 2022] PUT /identity/v3/projects/52837239ebfd465ebaccc583f4f1b1ac/users/20166c4e5f34486898274460d1f5bbaf/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:13.473448 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d594c0-17d9-4797-a48d-28ad7bd19354 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:13.500904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 823/1630] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:13.514451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f338cd-cba4-4490-8919-0f226b1894b2 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:13.578854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:13.612169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f338cd-cba4-4490-8919-0f226b1894b2 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:13.620128 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d594c0-17d9-4797-a48d-28ad7bd19354 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:13.633568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e93f85-f0c1-462c-b29f-66347bb043df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:13.634552 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e93f85-f0c1-462c-b29f-66347bb043df admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:13.635030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e93f85-f0c1-462c-b29f-66347bb043df admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:13.635462 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e93f85-f0c1-462c-b29f-66347bb043df admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:13.636384 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e93f85-f0c1-462c-b29f-66347bb043df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:13.638797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e93f85-f0c1-462c-b29f-66347bb043df admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:13.667188 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 808/1631] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:30:13.672601 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e93f85-f0c1-462c-b29f-66347bb043df admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:13.698790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e93f85-f0c1-462c-b29f-66347bb043df admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:13.706861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a05af1-bc4c-4a5c-a196-1c938e98b112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:13.708151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a05af1-bc4c-4a5c-a196-1c938e98b112 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:13.708654 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a05af1-bc4c-4a5c-a196-1c938e98b112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:13.709162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a05af1-bc4c-4a5c-a196-1c938e98b112 admin admin] PATH_INFO: `/v3/projects/52837239ebfd465ebaccc583f4f1b1ac/users/20166c4e5f34486898274460d1f5bbaf/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:13.716883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a05af1-bc4c-4a5c-a196-1c938e98b112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52837239ebfd465ebaccc583f4f1b1ac, user_id=20166c4e5f34486898274460d1f5bbaf, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:13.719931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a05af1-bc4c-4a5c-a196-1c938e98b112 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:13.731564 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e815c87-7da7-4f44-a33a-e3b211afda67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:13.732294 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e815c87-7da7-4f44-a33a-e3b211afda67 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:13.732585 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e815c87-7da7-4f44-a33a-e3b211afda67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:13.732866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e815c87-7da7-4f44-a33a-e3b211afda67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:13.742681 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 809/1632] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:13 2022] PUT /identity/v3/projects/52837239ebfd465ebaccc583f4f1b1ac/users/20166c4e5f34486898274460d1f5bbaf/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:13.754017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70e3b4-5adb-4ee4-9d27-80915decd793 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:13.754549 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70e3b4-5adb-4ee4-9d27-80915decd793 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:13.755090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70e3b4-5adb-4ee4-9d27-80915decd793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:13.843297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:13.847604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-df70e3b4-5adb-4ee4-9d27-80915decd793 None None] MFA Rules not processed for user `20166c4e5f34486898274460d1f5bbaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:13.848818 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-df70e3b4-5adb-4ee4-9d27-80915decd793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:13.914663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e815c87-7da7-4f44-a33a-e3b211afda67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:13.916814 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e815c87-7da7-4f44-a33a-e3b211afda67 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:13.949002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 824/1633] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:13.951603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f75c29-8736-430f-b00c-30ea1ad7e210 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:13.951898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f75c29-8736-430f-b00c-30ea1ad7e210 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:13.952184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f75c29-8736-430f-b00c-30ea1ad7e210 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:13.956253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 825/1634] 10.222.0.42 () {54 vars in 912 bytes} [Tue Jul 19 15:30:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:30:13.961785 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a9472-1dc4-4d6c-9a12-210530cba1eb None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:13.963108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a9472-1dc4-4d6c-9a12-210530cba1eb None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:13.963108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a9472-1dc4-4d6c-9a12-210530cba1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:13.988270 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-d14a9472-1dc4-4d6c-9a12-210530cba1eb None None] MFA Rules not processed for user `bff6a2d4bc734904b2615608a86b097d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:13.993877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d14a9472-1dc4-4d6c-9a12-210530cba1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:14.063938 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 810/1635] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:30:14.084192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70e3b4-5adb-4ee4-9d27-80915decd793 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:14.117525 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 826/1636] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:30:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:14.172447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a9472-1dc4-4d6c-9a12-210530cba1eb None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:14.244861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70e3b4-5adb-4ee4-9d27-80915decd793 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:14.257512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcfdf7-71b5-44b6-bcdd-cc2179123a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:14.259018 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcfdf7-71b5-44b6-bcdd-cc2179123a2b service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:14.259428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcfdf7-71b5-44b6-bcdd-cc2179123a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:14.259838 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcfdf7-71b5-44b6-bcdd-cc2179123a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:14.298587 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fcfdf7-71b5-44b6-bcdd-cc2179123a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:14.300549 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fcfdf7-71b5-44b6-bcdd-cc2179123a2b service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:14.338863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a9472-1dc4-4d6c-9a12-210530cba1eb None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:14.349589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 811/1637] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:14.350444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9834f5-3695-4561-9e0b-a6fe3177f7ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:14.351298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9834f5-3695-4561-9e0b-a6fe3177f7ba service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:14.351691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9834f5-3695-4561-9e0b-a6fe3177f7ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:14.352073 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9834f5-3695-4561-9e0b-a6fe3177f7ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:14.369934 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9834f5-3695-4561-9e0b-a6fe3177f7ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:14.371707 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9834f5-3695-4561-9e0b-a6fe3177f7ba service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:14.415356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 827/1638] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:30:15.027589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcfdf7-71b5-44b6-bcdd-cc2179123a2b service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:15.038445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcfdf7-71b5-44b6-bcdd-cc2179123a2b service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:15.052879 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4ebf8-d69b-4a17-8ce5-dd05e096aa43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:15.053825 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4ebf8-d69b-4a17-8ce5-dd05e096aa43 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:15.054109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4ebf8-d69b-4a17-8ce5-dd05e096aa43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:15.054385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4ebf8-d69b-4a17-8ce5-dd05e096aa43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:15.272639 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:15.315203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4ebf8-d69b-4a17-8ce5-dd05e096aa43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:15.316689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4ebf8-d69b-4a17-8ce5-dd05e096aa43 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:15.349571 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 812/1639] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:16.189819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9834f5-3695-4561-9e0b-a6fe3177f7ba service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:16.313565 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9834f5-3695-4561-9e0b-a6fe3177f7ba service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:16.324393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20a192-1ca3-459d-903a-1ae07ccf6c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:16.325744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20a192-1ca3-459d-903a-1ae07ccf6c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:16.326170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20a192-1ca3-459d-903a-1ae07ccf6c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:16.326594 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20a192-1ca3-459d-903a-1ae07ccf6c8d admin admin] PATH_INFO: `/v3/users/3b7365e66adc407cbe719e5bd5558008` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:16.331608 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20a192-1ca3-459d-903a-1ae07ccf6c8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b7365e66adc407cbe719e5bd5558008)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:16.333903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20a192-1ca3-459d-903a-1ae07ccf6c8d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:16.336420 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:16.439348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-cc20a192-1ca3-459d-903a-1ae07ccf6c8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b7365e66adc407cbe719e5bd5558008'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:16.464207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-cc20a192-1ca3-459d-903a-1ae07ccf6c8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b7365e66adc407cbe719e5bd5558008'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:16.482854 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-cc20a192-1ca3-459d-903a-1ae07ccf6c8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b7365e66adc407cbe719e5bd5558008'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:16.500935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-cc20a192-1ca3-459d-903a-1ae07ccf6c8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b7365e66adc407cbe719e5bd5558008'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:16.501874 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-cc20a192-1ca3-459d-903a-1ae07ccf6c8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b7365e66adc407cbe719e5bd5558008'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:16.524197 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a73801-8ab3-4baa-abd3-e7ab428ccf1f None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:16.526027 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a73801-8ab3-4baa-abd3-e7ab428ccf1f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:16.526629 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a73801-8ab3-4baa-abd3-e7ab428ccf1f None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:16.529210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 813/1640] 10.222.0.42 () {54 vars in 962 bytes} [Tue Jul 19 15:30:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:30:16.537415 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 828/1641] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:16 2022] DELETE /identity/v3/users/3b7365e66adc407cbe719e5bd5558008 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:16.542668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653736f-dd84-4be5-a497-f80f5bf9d1a0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:16.543117 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653736f-dd84-4be5-a497-f80f5bf9d1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:16.543572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653736f-dd84-4be5-a497-f80f5bf9d1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:16.547822 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64c8b7-ebe5-4f29-9d4e-3c6b9316bd14 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:16.549555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64c8b7-ebe5-4f29-9d4e-3c6b9316bd14 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:16.550162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64c8b7-ebe5-4f29-9d4e-3c6b9316bd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:16.622042 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c653736f-dd84-4be5-a497-f80f5bf9d1a0 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:16.623632 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c653736f-dd84-4be5-a497-f80f5bf9d1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:16.637404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-dd64c8b7-ebe5-4f29-9d4e-3c6b9316bd14 None None] MFA Rules not processed for user `bff6a2d4bc734904b2615608a86b097d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:16.641476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-dd64c8b7-ebe5-4f29-9d4e-3c6b9316bd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:16.771837 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 814/1642] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:16.807743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8448b4d-73d9-4d78-9e52-03e9399cd183 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:16.808164 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8448b4d-73d9-4d78-9e52-03e9399cd183 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:16.808541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8448b4d-73d9-4d78-9e52-03e9399cd183 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:16.809979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 829/1643] 10.222.0.42 () {58 vars in 1059 bytes} [Tue Jul 19 15:30:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:16.810990 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 815/1644] 10.222.0.42 () {54 vars in 912 bytes} [Tue Jul 19 15:30:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 15:30:16.816915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a14510-39d6-462e-add1-c2c4fd01a552 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:16.817538 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a14510-39d6-462e-add1-c2c4fd01a552 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:16.817919 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a14510-39d6-462e-add1-c2c4fd01a552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:16.821705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8448b4d-73d9-4d78-9e52-03e9399cd183 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:16.824397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a8448b4d-73d9-4d78-9e52-03e9399cd183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:16.859727 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-49a14510-39d6-462e-add1-c2c4fd01a552 None None] MFA Rules not processed for user `bff6a2d4bc734904b2615608a86b097d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:16.861567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-49a14510-39d6-462e-add1-c2c4fd01a552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:16.877477 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 830/1645] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:30:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:16.894199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a14510-39d6-462e-add1-c2c4fd01a552 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:16.898473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-49a14510-39d6-462e-add1-c2c4fd01a552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:16.954506 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8448b4d-73d9-4d78-9e52-03e9399cd183 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:16.959772 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0089921-d37c-4c02-b987-0dfec31c6661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:16.961572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0089921-d37c-4c02-b987-0dfec31c6661 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:16.962018 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0089921-d37c-4c02-b987-0dfec31c6661 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:16.962608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0089921-d37c-4c02-b987-0dfec31c6661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:16.979925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0089921-d37c-4c02-b987-0dfec31c6661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:16.983129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0089921-d37c-4c02-b987-0dfec31c6661 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:17.001371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 816/1646] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:30:17.003272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a14510-39d6-462e-add1-c2c4fd01a552 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:17.010669 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98f93b-db33-4f19-86b7-0b947acfdf3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:17.011568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98f93b-db33-4f19-86b7-0b947acfdf3d service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:17.011967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98f93b-db33-4f19-86b7-0b947acfdf3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:17.012350 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98f93b-db33-4f19-86b7-0b947acfdf3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:17.023687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b98f93b-db33-4f19-86b7-0b947acfdf3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:17.025780 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b98f93b-db33-4f19-86b7-0b947acfdf3d service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:17.038453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 831/1647] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:17.238118 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81609065-8455-46c2-9c6e-ffcf391d7f6e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:17.238507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81609065-8455-46c2-9c6e-ffcf391d7f6e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:17.238933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81609065-8455-46c2-9c6e-ffcf391d7f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:17.303230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-81609065-8455-46c2-9c6e-ffcf391d7f6e None None] MFA Rules not processed for user `46cd0aa86f274786b53c492a72c199cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:17.304938 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-81609065-8455-46c2-9c6e-ffcf391d7f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:17.429929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 817/1648] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:17.430141 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98f93b-db33-4f19-86b7-0b947acfdf3d service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:17.431715 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2b98f93b-db33-4f19-86b7-0b947acfdf3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:17.450323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98f93b-db33-4f19-86b7-0b947acfdf3d service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:17.460662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d74a50-d733-4d85-9c3f-aa4ec3581c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:17.464875 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d74a50-d733-4d85-9c3f-aa4ec3581c32 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:17.465309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d74a50-d733-4d85-9c3f-aa4ec3581c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:17.465702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d74a50-d733-4d85-9c3f-aa4ec3581c32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:17.466646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d74a50-d733-4d85-9c3f-aa4ec3581c32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:17.469612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d74a50-d733-4d85-9c3f-aa4ec3581c32 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:17.476287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:17.524299 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 832/1649] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:17 2022] POST /identity/v3/projects => generated 405 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:17.529705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81609065-8455-46c2-9c6e-ffcf391d7f6e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:17.548963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81609065-8455-46c2-9c6e-ffcf391d7f6e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:17.585109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:17.764170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a117710-c9f0-4cb9-b7f7-e09de2765531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:17.764885 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a117710-c9f0-4cb9-b7f7-e09de2765531 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:17.765277 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a117710-c9f0-4cb9-b7f7-e09de2765531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:17.765473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a117710-c9f0-4cb9-b7f7-e09de2765531 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:17.766398 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a117710-c9f0-4cb9-b7f7-e09de2765531 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:17.768466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a117710-c9f0-4cb9-b7f7-e09de2765531 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:17.820413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 818/1650] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:17 2022] POST /identity/v3/users => generated 309 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:17.823197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d74a50-d733-4d85-9c3f-aa4ec3581c32 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:17.844203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d74a50-d733-4d85-9c3f-aa4ec3581c32 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:17.852299 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ca1d6-678e-469b-a717-fb93e8b4ea18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:17.853054 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ca1d6-678e-469b-a717-fb93e8b4ea18 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:17.853367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ca1d6-678e-469b-a717-fb93e8b4ea18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:17.853769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ca1d6-678e-469b-a717-fb93e8b4ea18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:17.854701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ca1d6-678e-469b-a717-fb93e8b4ea18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:17.856704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ca1d6-678e-469b-a717-fb93e8b4ea18 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:17.877608 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 833/1651] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:17.882399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a117710-c9f0-4cb9-b7f7-e09de2765531 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:17.904370 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a117710-c9f0-4cb9-b7f7-e09de2765531 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:17.920593 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbe383-fc64-410a-950c-7e2914e32666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:17.921541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbe383-fc64-410a-950c-7e2914e32666 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:17.921831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbe383-fc64-410a-950c-7e2914e32666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:17.922113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbe383-fc64-410a-950c-7e2914e32666 admin admin] PATH_INFO: `/v3/projects/62a22e9e1669461b9ec9e03a523bf7f8/users/36b87ad6b4fb48f09393d9c561371fd1/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:17.967283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbe383-fc64-410a-950c-7e2914e32666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a22e9e1669461b9ec9e03a523bf7f8, user_id=36b87ad6b4fb48f09393d9c561371fd1, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:17.970431 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbe383-fc64-410a-950c-7e2914e32666 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:18.004776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 819/1652] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:17 2022] PUT /identity/v3/projects/62a22e9e1669461b9ec9e03a523bf7f8/users/36b87ad6b4fb48f09393d9c561371fd1/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:18.008552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ca1d6-678e-469b-a717-fb93e8b4ea18 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:18.037215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ca1d6-678e-469b-a717-fb93e8b4ea18 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:18.051158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbe383-fc64-410a-950c-7e2914e32666 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:18.054527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-39bbe383-fc64-410a-950c-7e2914e32666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:18.215188 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b5ff8-ec0d-4302-818e-79e6200a48b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:18.216707 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b5ff8-ec0d-4302-818e-79e6200a48b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:18.217128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b5ff8-ec0d-4302-818e-79e6200a48b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:18.217563 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b5ff8-ec0d-4302-818e-79e6200a48b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:18.218458 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b5ff8-ec0d-4302-818e-79e6200a48b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:18.220207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b5ff8-ec0d-4302-818e-79e6200a48b4 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:18.240765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 834/1653] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:18.245173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbe383-fc64-410a-950c-7e2914e32666 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:18.246229 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b5ff8-ec0d-4302-818e-79e6200a48b4 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:18.260777 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b5ff8-ec0d-4302-818e-79e6200a48b4 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:18.261648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0d279-b6c1-4a88-a6ee-44aef0283993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:18.262837 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0d279-b6c1-4a88-a6ee-44aef0283993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:18.263252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0d279-b6c1-4a88-a6ee-44aef0283993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:18.263743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0d279-b6c1-4a88-a6ee-44aef0283993 admin admin] PATH_INFO: `/v3/projects/2e61067f09b04588ac46e5f74a29a22f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:18.265832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0d279-b6c1-4a88-a6ee-44aef0283993 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e61067f09b04588ac46e5f74a29a22f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:18.267914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0d279-b6c1-4a88-a6ee-44aef0283993 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:18.280459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c8f22-5385-4d50-aee7-a27c6edee206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:18.282353 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c8f22-5385-4d50-aee7-a27c6edee206 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:18.282764 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c8f22-5385-4d50-aee7-a27c6edee206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:18.283317 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c8f22-5385-4d50-aee7-a27c6edee206 admin admin] PATH_INFO: `/v3/projects/62a22e9e1669461b9ec9e03a523bf7f8/users/36b87ad6b4fb48f09393d9c561371fd1/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:18.287027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c8f22-5385-4d50-aee7-a27c6edee206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a22e9e1669461b9ec9e03a523bf7f8, user_id=36b87ad6b4fb48f09393d9c561371fd1, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:18.289954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c8f22-5385-4d50-aee7-a27c6edee206 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:18.315108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 835/1654] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:18 2022] PUT /identity/v3/projects/62a22e9e1669461b9ec9e03a523bf7f8/users/36b87ad6b4fb48f09393d9c561371fd1/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:18.373778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cda0d279-b6c1-4a88-a6ee-44aef0283993 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e61067f09b04588ac46e5f74a29a22f', 'request_id': 'req-cda0d279-b6c1-4a88-a6ee-44aef0283993'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:18.390068 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cda0d279-b6c1-4a88-a6ee-44aef0283993 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e61067f09b04588ac46e5f74a29a22f', 'request_id': 'req-cda0d279-b6c1-4a88-a6ee-44aef0283993'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:18.402298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cda0d279-b6c1-4a88-a6ee-44aef0283993 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e61067f09b04588ac46e5f74a29a22f', 'request_id': 'req-cda0d279-b6c1-4a88-a6ee-44aef0283993'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:18.423690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cda0d279-b6c1-4a88-a6ee-44aef0283993 admin admin] The token cache is being invalidate because project 2e61067f09b04588ac46e5f74a29a22f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:18.424205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cda0d279-b6c1-4a88-a6ee-44aef0283993 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:18.425056 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cda0d279-b6c1-4a88-a6ee-44aef0283993 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:18.430745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 820/1655] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:18 2022] DELETE /identity/v3/projects/2e61067f09b04588ac46e5f74a29a22f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:18.431150 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c8f22-5385-4d50-aee7-a27c6edee206 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:18.433603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1c0c8f22-5385-4d50-aee7-a27c6edee206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:18.562794 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c8f22-5385-4d50-aee7-a27c6edee206 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:18.572692 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0d279-b6c1-4a88-a6ee-44aef0283993 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:18.574334 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cda0d279-b6c1-4a88-a6ee-44aef0283993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:18.631616 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:18.736109 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f37dc-36c2-4a6f-9b17-be5ae8dd2cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:18.736880 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f37dc-36c2-4a6f-9b17-be5ae8dd2cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:18.737190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f37dc-36c2-4a6f-9b17-be5ae8dd2cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:18.737484 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f37dc-36c2-4a6f-9b17-be5ae8dd2cdf admin admin] PATH_INFO: `/v3/users/97668cce1d0e45ba936f651393e7a794` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:18.737965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0d279-b6c1-4a88-a6ee-44aef0283993 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:18.739097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f37dc-36c2-4a6f-9b17-be5ae8dd2cdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=97668cce1d0e45ba936f651393e7a794)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:18.743223 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f37dc-36c2-4a6f-9b17-be5ae8dd2cdf admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:18.744145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039829b4-1727-4e07-9e4b-1d1c8d05a2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:18.747179 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039829b4-1727-4e07-9e4b-1d1c8d05a2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:18.747586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039829b4-1727-4e07-9e4b-1d1c8d05a2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:18.747970 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039829b4-1727-4e07-9e4b-1d1c8d05a2b9 admin admin] PATH_INFO: `/v3/users/b9dfaf168adf45618c828806ec285026` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:18.749465 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039829b4-1727-4e07-9e4b-1d1c8d05a2b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9dfaf168adf45618c828806ec285026)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:18.750617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-039829b4-1727-4e07-9e4b-1d1c8d05a2b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 15:30:18.750617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) Jul 19 15:30:18.750617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: Jul 19 15:30:18.752505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039829b4-1727-4e07-9e4b-1d1c8d05a2b9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:18.757236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:18.846915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-303f37dc-36c2-4a6f-9b17-be5ae8dd2cdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97668cce1d0e45ba936f651393e7a794'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:18.902532 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-303f37dc-36c2-4a6f-9b17-be5ae8dd2cdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97668cce1d0e45ba936f651393e7a794'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:18.906479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-303f37dc-36c2-4a6f-9b17-be5ae8dd2cdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97668cce1d0e45ba936f651393e7a794'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:18.908364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-039829b4-1727-4e07-9e4b-1d1c8d05a2b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9dfaf168adf45618c828806ec285026'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:18.941380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-303f37dc-36c2-4a6f-9b17-be5ae8dd2cdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97668cce1d0e45ba936f651393e7a794'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:18.952154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-303f37dc-36c2-4a6f-9b17-be5ae8dd2cdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97668cce1d0e45ba936f651393e7a794'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:18.981130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 836/1656] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:18 2022] DELETE /identity/v3/users/97668cce1d0e45ba936f651393e7a794 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:18.987110 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-039829b4-1727-4e07-9e4b-1d1c8d05a2b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9dfaf168adf45618c828806ec285026'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:18.988112 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-039829b4-1727-4e07-9e4b-1d1c8d05a2b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9dfaf168adf45618c828806ec285026'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:18.990393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-039829b4-1727-4e07-9e4b-1d1c8d05a2b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9dfaf168adf45618c828806ec285026'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:19.009683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-039829b4-1727-4e07-9e4b-1d1c8d05a2b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9dfaf168adf45618c828806ec285026'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:19.056774 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 821/1657] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:18 2022] DELETE /identity/v3/users/b9dfaf168adf45618c828806ec285026 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:19.060335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9418c7-05fd-4b2a-ad40-c4193b8ca9f5 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:19.060632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9418c7-05fd-4b2a-ad40-c4193b8ca9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:19.060910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9418c7-05fd-4b2a-ad40-c4193b8ca9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:19.083565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039829b4-1727-4e07-9e4b-1d1c8d05a2b9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:19.085095 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-039829b4-1727-4e07-9e4b-1d1c8d05a2b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:19.106106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-bf9418c7-05fd-4b2a-ad40-c4193b8ca9f5 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:19.108156 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-bf9418c7-05fd-4b2a-ad40-c4193b8ca9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:19.243377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 837/1658] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:19.246188 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039829b4-1727-4e07-9e4b-1d1c8d05a2b9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:19.259303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bbb4f3-1d65-4f27-9efd-a91e1d643b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:19.261446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbb4f3-1d65-4f27-9efd-a91e1d643b65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:19.261975 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbb4f3-1d65-4f27-9efd-a91e1d643b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:19.262529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbb4f3-1d65-4f27-9efd-a91e1d643b65 admin admin] PATH_INFO: `/v3/users/2123408c9dc0442b83a36e95efa04ee8` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:19.264167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bbb4f3-1d65-4f27-9efd-a91e1d643b65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2123408c9dc0442b83a36e95efa04ee8)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:19.266545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bbb4f3-1d65-4f27-9efd-a91e1d643b65 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:19.284855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9418c7-05fd-4b2a-ad40-c4193b8ca9f5 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:19.293377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-bf9418c7-05fd-4b2a-ad40-c4193b8ca9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:19.333454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:19.342150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-31bbb4f3-1d65-4f27-9efd-a91e1d643b65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2123408c9dc0442b83a36e95efa04ee8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:19.374448 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-31bbb4f3-1d65-4f27-9efd-a91e1d643b65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2123408c9dc0442b83a36e95efa04ee8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:19.380124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-31bbb4f3-1d65-4f27-9efd-a91e1d643b65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2123408c9dc0442b83a36e95efa04ee8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:19.380897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-31bbb4f3-1d65-4f27-9efd-a91e1d643b65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2123408c9dc0442b83a36e95efa04ee8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:19.401673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-31bbb4f3-1d65-4f27-9efd-a91e1d643b65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2123408c9dc0442b83a36e95efa04ee8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:19.442712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 822/1659] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:19 2022] DELETE /identity/v3/users/2123408c9dc0442b83a36e95efa04ee8 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:30:19.447042 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2f395-fed6-4f2e-846c-96ab2b4099cf None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:19.448101 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2f395-fed6-4f2e-846c-96ab2b4099cf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:19.449179 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2f395-fed6-4f2e-846c-96ab2b4099cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:19.489916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1ab2f395-fed6-4f2e-846c-96ab2b4099cf None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:19.491304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1ab2f395-fed6-4f2e-846c-96ab2b4099cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:19.519055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9418c7-05fd-4b2a-ad40-c4193b8ca9f5 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:19.684578 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 823/1660] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:19.732069 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2f395-fed6-4f2e-846c-96ab2b4099cf None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:19.734700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1ab2f395-fed6-4f2e-846c-96ab2b4099cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:19.752710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:19.784736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19b050-c50d-42c5-a6a5-481bc5edbaa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:19.785761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19b050-c50d-42c5-a6a5-481bc5edbaa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:19.786901 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19b050-c50d-42c5-a6a5-481bc5edbaa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:19.787380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19b050-c50d-42c5-a6a5-481bc5edbaa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:19.792543 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7c19b050-c50d-42c5-a6a5-481bc5edbaa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:19.826733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19b050-c50d-42c5-a6a5-481bc5edbaa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:19.829494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19b050-c50d-42c5-a6a5-481bc5edbaa2 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:19.855745 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 838/1661] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:19.876734 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2f395-fed6-4f2e-846c-96ab2b4099cf None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:19.880959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b08b5-9eb2-4e60-8602-484eae74068c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:19.881665 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b08b5-9eb2-4e60-8602-484eae74068c service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:19.881952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b08b5-9eb2-4e60-8602-484eae74068c service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:19.882231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b08b5-9eb2-4e60-8602-484eae74068c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:19.908248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b08b5-9eb2-4e60-8602-484eae74068c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:19.909916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b08b5-9eb2-4e60-8602-484eae74068c service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:19.930770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 824/1662] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:21.063660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19b050-c50d-42c5-a6a5-481bc5edbaa2 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:21.066655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7c19b050-c50d-42c5-a6a5-481bc5edbaa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:21.106030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19b050-c50d-42c5-a6a5-481bc5edbaa2 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:21.130427 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:21.131489 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:21.131797 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:21.132067 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e admin admin] PATH_INFO: `/v3/projects/58a8a875a16c46ecb7a0d4d845183ba4` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:21.136904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e admin admin] RBAC: Authorizing `identity:delete_project(project_id=58a8a875a16c46ecb7a0d4d845183ba4)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:21.138472 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: WARNING py.warnings [None req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 15:30:21.138472 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: warnings.warn(msg) Jul 19 15:30:21.138472 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: Jul 19 15:30:21.139637 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:21.345667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58a8a875a16c46ecb7a0d4d845183ba4', 'request_id': 'req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:21.353598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58a8a875a16c46ecb7a0d4d845183ba4', 'request_id': 'req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:21.392340 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58a8a875a16c46ecb7a0d4d845183ba4', 'request_id': 'req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:21.419409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e admin admin] The token cache is being invalidate because project 58a8a875a16c46ecb7a0d4d845183ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:21.419916 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:21.422378 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a74b17d2-14a6-4d21-8e21-06c0c4ff190e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:21.425048 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 839/1663] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:21 2022] DELETE /identity/v3/projects/58a8a875a16c46ecb7a0d4d845183ba4 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:21.433613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eab2b3-39b8-4b81-afdd-6e3df84f9f0e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:21.434158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eab2b3-39b8-4b81-afdd-6e3df84f9f0e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:21.434158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eab2b3-39b8-4b81-afdd-6e3df84f9f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:21.484767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-87eab2b3-39b8-4b81-afdd-6e3df84f9f0e None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:21.489669 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-87eab2b3-39b8-4b81-afdd-6e3df84f9f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:21.715211 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 825/1664] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:21.718708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00907f30-11f9-4d37-b8aa-8962909f8927 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:21.719798 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00907f30-11f9-4d37-b8aa-8962909f8927 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:21.720280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00907f30-11f9-4d37-b8aa-8962909f8927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:21.739660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eab2b3-39b8-4b81-afdd-6e3df84f9f0e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:21.740996 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-87eab2b3-39b8-4b81-afdd-6e3df84f9f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:21.754652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:21.773325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-00907f30-11f9-4d37-b8aa-8962909f8927 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:21.775700 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-00907f30-11f9-4d37-b8aa-8962909f8927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:21.780696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eab2b3-39b8-4b81-afdd-6e3df84f9f0e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:21.802712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 840/1665] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:21.806950 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00907f30-11f9-4d37-b8aa-8962909f8927 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:21.807841 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd3735-ecd2-4b83-b24e-3df79c542eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:21.808766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd3735-ecd2-4b83-b24e-3df79c542eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:21.809042 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd3735-ecd2-4b83-b24e-3df79c542eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:21.809478 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd3735-ecd2-4b83-b24e-3df79c542eef admin admin] PATH_INFO: `/v3/projects/8c457bf7357846e59a4db9c3492463a5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:21.812867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd3735-ecd2-4b83-b24e-3df79c542eef admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c457bf7357846e59a4db9c3492463a5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:21.815011 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd3735-ecd2-4b83-b24e-3df79c542eef admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:21.822532 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00907f30-11f9-4d37-b8aa-8962909f8927 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:21.835870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc4bf3-1c4d-4467-aeed-9886c0450b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:21.836626 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc4bf3-1c4d-4467-aeed-9886c0450b53 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:21.836910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc4bf3-1c4d-4467-aeed-9886c0450b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:21.837293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc4bf3-1c4d-4467-aeed-9886c0450b53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:21.838614 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc4bf3-1c4d-4467-aeed-9886c0450b53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:21.838905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc4bf3-1c4d-4467-aeed-9886c0450b53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:21.841130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc4bf3-1c4d-4467-aeed-9886c0450b53 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:21.879590 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 841/1666] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:30:21.882450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc4bf3-1c4d-4467-aeed-9886c0450b53 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:21.915548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc4bf3-1c4d-4467-aeed-9886c0450b53 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:21.948033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23cd3735-ecd2-4b83-b24e-3df79c542eef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c457bf7357846e59a4db9c3492463a5', 'request_id': 'req-23cd3735-ecd2-4b83-b24e-3df79c542eef'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:21.980439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23cd3735-ecd2-4b83-b24e-3df79c542eef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c457bf7357846e59a4db9c3492463a5', 'request_id': 'req-23cd3735-ecd2-4b83-b24e-3df79c542eef'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:22.007193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23cd3735-ecd2-4b83-b24e-3df79c542eef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c457bf7357846e59a4db9c3492463a5', 'request_id': 'req-23cd3735-ecd2-4b83-b24e-3df79c542eef'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:22.041092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23cd3735-ecd2-4b83-b24e-3df79c542eef admin admin] The token cache is being invalidate because project 8c457bf7357846e59a4db9c3492463a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:22.041477 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23cd3735-ecd2-4b83-b24e-3df79c542eef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:22.043240 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23cd3735-ecd2-4b83-b24e-3df79c542eef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:22.066210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 826/1667] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:21 2022] DELETE /identity/v3/projects/8c457bf7357846e59a4db9c3492463a5 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:22.070109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd3735-ecd2-4b83-b24e-3df79c542eef admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:22.078007 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-23cd3735-ecd2-4b83-b24e-3df79c542eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:22.142740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fe1fc-30ff-4298-92c2-206db422482e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:22.144059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fe1fc-30ff-4298-92c2-206db422482e admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:22.145536 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fe1fc-30ff-4298-92c2-206db422482e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:22.146057 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fe1fc-30ff-4298-92c2-206db422482e admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:22.146999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fe1fc-30ff-4298-92c2-206db422482e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:22.152351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fe1fc-30ff-4298-92c2-206db422482e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:22.232554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 842/1668] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:21 2022] POST /identity/v3/projects => generated 379 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:22.236922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fe1fc-30ff-4298-92c2-206db422482e admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:22.239281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3e8fe1fc-30ff-4298-92c2-206db422482e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:22.324517 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:22.339699 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd3735-ecd2-4b83-b24e-3df79c542eef admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:22.422292 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fe1fc-30ff-4298-92c2-206db422482e admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:22.429326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb4770-2634-4e8e-bc22-5317d3ecf2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:22.430362 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb4770-2634-4e8e-bc22-5317d3ecf2ba admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:22.444995 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb4770-2634-4e8e-bc22-5317d3ecf2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:22.445635 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb4770-2634-4e8e-bc22-5317d3ecf2ba admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:22.446622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb4770-2634-4e8e-bc22-5317d3ecf2ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:22.450158 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb4770-2634-4e8e-bc22-5317d3ecf2ba admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:22.485849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6889570-f2fd-45b4-aa39-422190775e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:22.490870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6889570-f2fd-45b4-aa39-422190775e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:22.491419 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6889570-f2fd-45b4-aa39-422190775e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:22.491845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6889570-f2fd-45b4-aa39-422190775e1d admin admin] PATH_INFO: `/v3/users/20166c4e5f34486898274460d1f5bbaf` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:22.493576 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6889570-f2fd-45b4-aa39-422190775e1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=20166c4e5f34486898274460d1f5bbaf)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:22.495867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 843/1669] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:22 2022] POST /identity/v3/users => generated 290 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:30:22.496855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb4770-2634-4e8e-bc22-5317d3ecf2ba admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:22.504208 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6889570-f2fd-45b4-aa39-422190775e1d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:22.508477 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-4bdb4770-2634-4e8e-bc22-5317d3ecf2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:22.536792 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb4770-2634-4e8e-bc22-5317d3ecf2ba admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:22.554636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4b72b-b6e4-44bb-82d2-7fbeb68ba325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:22.555890 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4b72b-b6e4-44bb-82d2-7fbeb68ba325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:22.561298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4b72b-b6e4-44bb-82d2-7fbeb68ba325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:22.561736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4b72b-b6e4-44bb-82d2-7fbeb68ba325 admin admin] PATH_INFO: `/v3/projects/732ecfbf08874525a7ecbdec9abf34eb` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:22.566474 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4b72b-b6e4-44bb-82d2-7fbeb68ba325 admin admin] RBAC: Authorizing `identity:delete_project(project_id=732ecfbf08874525a7ecbdec9abf34eb)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:22.569630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4b72b-b6e4-44bb-82d2-7fbeb68ba325 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:22.580817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:22.623265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e6889570-f2fd-45b4-aa39-422190775e1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20166c4e5f34486898274460d1f5bbaf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:22.659846 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e6889570-f2fd-45b4-aa39-422190775e1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20166c4e5f34486898274460d1f5bbaf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:22.660514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e6889570-f2fd-45b4-aa39-422190775e1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20166c4e5f34486898274460d1f5bbaf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:22.661953 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e6889570-f2fd-45b4-aa39-422190775e1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20166c4e5f34486898274460d1f5bbaf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:22.695753 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e6889570-f2fd-45b4-aa39-422190775e1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20166c4e5f34486898274460d1f5bbaf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:22.718428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 827/1670] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:22 2022] DELETE /identity/v3/users/20166c4e5f34486898274460d1f5bbaf => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:30:22.719320 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6889570-f2fd-45b4-aa39-422190775e1d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:22.724170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e6889570-f2fd-45b4-aa39-422190775e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:22.850188 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d3a4b72b-b6e4-44bb-82d2-7fbeb68ba325 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '732ecfbf08874525a7ecbdec9abf34eb', 'request_id': 'req-d3a4b72b-b6e4-44bb-82d2-7fbeb68ba325'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:22.873916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6889570-f2fd-45b4-aa39-422190775e1d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:22.881323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d3a4b72b-b6e4-44bb-82d2-7fbeb68ba325 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '732ecfbf08874525a7ecbdec9abf34eb', 'request_id': 'req-d3a4b72b-b6e4-44bb-82d2-7fbeb68ba325'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:22.915561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d3a4b72b-b6e4-44bb-82d2-7fbeb68ba325 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '732ecfbf08874525a7ecbdec9abf34eb', 'request_id': 'req-d3a4b72b-b6e4-44bb-82d2-7fbeb68ba325'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:22.958739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d3a4b72b-b6e4-44bb-82d2-7fbeb68ba325 admin admin] The token cache is being invalidate because project 732ecfbf08874525a7ecbdec9abf34eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:22.962428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d3a4b72b-b6e4-44bb-82d2-7fbeb68ba325 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:22.964601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d3a4b72b-b6e4-44bb-82d2-7fbeb68ba325 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:22.967372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 844/1671] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:22 2022] DELETE /identity/v3/projects/732ecfbf08874525a7ecbdec9abf34eb => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:22.972962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e79e7b-165c-4eb1-93b3-ecd3014db0b9 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:22.973355 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e79e7b-165c-4eb1-93b3-ecd3014db0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:22.973638 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e79e7b-165c-4eb1-93b3-ecd3014db0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:23.031493 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-c5e79e7b-165c-4eb1-93b3-ecd3014db0b9 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:23.033382 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c5e79e7b-165c-4eb1-93b3-ecd3014db0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:23.068882 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d119e3-5f4a-43f8-b071-d1568f351930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:23.074486 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d119e3-5f4a-43f8-b071-d1568f351930 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:23.074694 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d119e3-5f4a-43f8-b071-d1568f351930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:23.074944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d119e3-5f4a-43f8-b071-d1568f351930 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:23.075733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d119e3-5f4a-43f8-b071-d1568f351930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:23.077514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d119e3-5f4a-43f8-b071-d1568f351930 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:23.110764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 828/1672] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:23.114698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d119e3-5f4a-43f8-b071-d1568f351930 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:23.116924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-36d119e3-5f4a-43f8-b071-d1568f351930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:23.131286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d119e3-5f4a-43f8-b071-d1568f351930 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:23.138229 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1432afa-37fd-4779-83c1-2c8b95ff99a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:23.138926 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1432afa-37fd-4779-83c1-2c8b95ff99a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:23.139213 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1432afa-37fd-4779-83c1-2c8b95ff99a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:23.139492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1432afa-37fd-4779-83c1-2c8b95ff99a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:23.140231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1432afa-37fd-4779-83c1-2c8b95ff99a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:23.141981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1432afa-37fd-4779-83c1-2c8b95ff99a6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:23.161547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 829/1673] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:30:23.168577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1432afa-37fd-4779-83c1-2c8b95ff99a6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:23.192633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1432afa-37fd-4779-83c1-2c8b95ff99a6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:23.210281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 845/1674] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:23.214583 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42462b8-3eea-4c4f-a9de-cb238ebead87 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:23.215433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb056e-a8af-45fd-98ad-cb1ed4935e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:23.216774 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42462b8-3eea-4c4f-a9de-cb238ebead87 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:23.216774 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42462b8-3eea-4c4f-a9de-cb238ebead87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:23.217587 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb056e-a8af-45fd-98ad-cb1ed4935e53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:23.217969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb056e-a8af-45fd-98ad-cb1ed4935e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:23.218276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb056e-a8af-45fd-98ad-cb1ed4935e53 admin admin] PATH_INFO: `/v3/projects/680475cbe61a44ba982cc0de3c0e301b/users/9f67b60fe38b4d86be9ad6a394336e9b/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:23.253051 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-b42462b8-3eea-4c4f-a9de-cb238ebead87 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:23.254311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b42462b8-3eea-4c4f-a9de-cb238ebead87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:23.257681 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb056e-a8af-45fd-98ad-cb1ed4935e53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=680475cbe61a44ba982cc0de3c0e301b, user_id=9f67b60fe38b4d86be9ad6a394336e9b, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:23.260649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb056e-a8af-45fd-98ad-cb1ed4935e53 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:23.284860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 846/1675] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:23.285715 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42462b8-3eea-4c4f-a9de-cb238ebead87 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:23.309618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 830/1676] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:23 2022] PUT /identity/v3/projects/680475cbe61a44ba982cc0de3c0e301b/users/9f67b60fe38b4d86be9ad6a394336e9b/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:23.320439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3215d6-5641-4e90-9003-352f9cb66342 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:23.321446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3215d6-5641-4e90-9003-352f9cb66342 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:23.327646 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3215d6-5641-4e90-9003-352f9cb66342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:23.339597 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42462b8-3eea-4c4f-a9de-cb238ebead87 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:23.406327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8b3215d6-5641-4e90-9003-352f9cb66342 None None] MFA Rules not processed for user `9f67b60fe38b4d86be9ad6a394336e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:23.407958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8b3215d6-5641-4e90-9003-352f9cb66342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:23.433535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:23.443593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:23.512548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c81c75-9b97-433d-aeab-8f002b34deec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:23.516755 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c81c75-9b97-433d-aeab-8f002b34deec admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:23.517043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c81c75-9b97-433d-aeab-8f002b34deec admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:23.517346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c81c75-9b97-433d-aeab-8f002b34deec admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:23.518119 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c81c75-9b97-433d-aeab-8f002b34deec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:23.518431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c81c75-9b97-433d-aeab-8f002b34deec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:23.522144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c81c75-9b97-433d-aeab-8f002b34deec admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:23.551533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 847/1677] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:30:23.558285 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c81c75-9b97-433d-aeab-8f002b34deec admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:23.570923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 831/1678] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:23.576449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3215d6-5641-4e90-9003-352f9cb66342 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:23.590759 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c81c75-9b97-433d-aeab-8f002b34deec admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:23.600615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3215d6-5641-4e90-9003-352f9cb66342 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:23.601477 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad946f-68d9-4863-9431-fa9debb1024a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:23.609231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8349d4a-f14c-420a-829e-d33a14d63402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:23.609552 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8349d4a-f14c-420a-829e-d33a14d63402 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:23.609849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8349d4a-f14c-420a-829e-d33a14d63402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:23.610131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8349d4a-f14c-420a-829e-d33a14d63402 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:23.611452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8349d4a-f14c-420a-829e-d33a14d63402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:23.614705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8349d4a-f14c-420a-829e-d33a14d63402 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:23.618500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad946f-68d9-4863-9431-fa9debb1024a admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:23.619104 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad946f-68d9-4863-9431-fa9debb1024a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:23.619656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad946f-68d9-4863-9431-fa9debb1024a admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:23.620523 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad946f-68d9-4863-9431-fa9debb1024a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:23.622885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad946f-68d9-4863-9431-fa9debb1024a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:23.661524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 832/1679] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:23 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:23.664799 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8349d4a-f14c-420a-829e-d33a14d63402 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:23.701275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 848/1680] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:23 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:23.705115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad946f-68d9-4863-9431-fa9debb1024a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:23.813314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad946f-68d9-4863-9431-fa9debb1024a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:23.838410 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8fa99-b62d-488f-85cb-8a69e6c4cda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:23.840126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8fa99-b62d-488f-85cb-8a69e6c4cda0 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:23.840869 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8fa99-b62d-488f-85cb-8a69e6c4cda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:23.841751 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8fa99-b62d-488f-85cb-8a69e6c4cda0 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:23.843464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8fa99-b62d-488f-85cb-8a69e6c4cda0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:23.850275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8fa99-b62d-488f-85cb-8a69e6c4cda0 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:23.852453 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8349d4a-f14c-420a-829e-d33a14d63402 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:23.869190 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77184106-f839-4f01-9a3a-5ab97284eae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:23.870847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77184106-f839-4f01-9a3a-5ab97284eae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:23.871205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77184106-f839-4f01-9a3a-5ab97284eae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:23.871500 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77184106-f839-4f01-9a3a-5ab97284eae6 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:23.874161 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77184106-f839-4f01-9a3a-5ab97284eae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:23.879766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77184106-f839-4f01-9a3a-5ab97284eae6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:23.884961 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 849/1681] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:23 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:30:23.888770 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8fa99-b62d-488f-85cb-8a69e6c4cda0 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:23.927210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 833/1682] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:23 2022] POST /identity/v3/users => generated 296 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:23.928594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77184106-f839-4f01-9a3a-5ab97284eae6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:23.935607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8fa99-b62d-488f-85cb-8a69e6c4cda0 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:23.954308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb241b19-f51d-4158-b155-4dcd1e12e315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:23.954629 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77184106-f839-4f01-9a3a-5ab97284eae6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:23.956664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb241b19-f51d-4158-b155-4dcd1e12e315 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:23.957316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb241b19-f51d-4158-b155-4dcd1e12e315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:23.958224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb241b19-f51d-4158-b155-4dcd1e12e315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:23.960258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb241b19-f51d-4158-b155-4dcd1e12e315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:23.967927 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb241b19-f51d-4158-b155-4dcd1e12e315 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:23.972138 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd18d29-136c-4df1-aa82-005f9de5468f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:23.976929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd18d29-136c-4df1-aa82-005f9de5468f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:23.977246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd18d29-136c-4df1-aa82-005f9de5468f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:23.977531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd18d29-136c-4df1-aa82-005f9de5468f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:23.978324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd18d29-136c-4df1-aa82-005f9de5468f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:23.989118 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd18d29-136c-4df1-aa82-005f9de5468f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:23.996923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 850/1683] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:30:24.001861 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb241b19-f51d-4158-b155-4dcd1e12e315 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:24.030359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 834/1684] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:24.032693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb241b19-f51d-4158-b155-4dcd1e12e315 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:24.034916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd18d29-136c-4df1-aa82-005f9de5468f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:24.045360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6dac09-760d-413c-99ec-db22dc51c62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:24.047380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6dac09-760d-413c-99ec-db22dc51c62f admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:24.048378 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6dac09-760d-413c-99ec-db22dc51c62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:24.049587 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6dac09-760d-413c-99ec-db22dc51c62f admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:24.051379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6dac09-760d-413c-99ec-db22dc51c62f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:24.052848 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd18d29-136c-4df1-aa82-005f9de5468f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:24.057799 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6dac09-760d-413c-99ec-db22dc51c62f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:24.060589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70582f8-8244-4809-9109-3a6ce1237551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:24.061351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70582f8-8244-4809-9109-3a6ce1237551 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:24.061645 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70582f8-8244-4809-9109-3a6ce1237551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:24.061929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70582f8-8244-4809-9109-3a6ce1237551 admin admin] PATH_INFO: `/v3/projects/56768634acf2480dab17ed84719f3573/users/7a308a7cdbc243c7a643dd25f22a18df/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:24.088712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 851/1685] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:24.090251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6dac09-760d-413c-99ec-db22dc51c62f admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:24.090381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70582f8-8244-4809-9109-3a6ce1237551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56768634acf2480dab17ed84719f3573, user_id=7a308a7cdbc243c7a643dd25f22a18df, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:24.095961 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70582f8-8244-4809-9109-3a6ce1237551 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:24.114420 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6dac09-760d-413c-99ec-db22dc51c62f admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:24.127280 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 835/1686] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:24 2022] PUT /identity/v3/projects/56768634acf2480dab17ed84719f3573/users/7a308a7cdbc243c7a643dd25f22a18df/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:24.131088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70582f8-8244-4809-9109-3a6ce1237551 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:24.243923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:24.286403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70582f8-8244-4809-9109-3a6ce1237551 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:24.293982 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44a350-6c0f-418a-bd46-30b379971f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:24.295822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44a350-6c0f-418a-bd46-30b379971f71 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:24.296171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44a350-6c0f-418a-bd46-30b379971f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:24.296578 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44a350-6c0f-418a-bd46-30b379971f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:24.297418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44a350-6c0f-418a-bd46-30b379971f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:24.299349 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44a350-6c0f-418a-bd46-30b379971f71 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:24.319676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 836/1687] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:24.321082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b71330-ec8d-4045-9acf-2ed25e90af5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:24.321561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc2b39-cc45-490f-96ba-b9c9800c74c4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:24.321561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc2b39-cc45-490f-96ba-b9c9800c74c4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:24.321738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc2b39-cc45-490f-96ba-b9c9800c74c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:24.322371 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b71330-ec8d-4045-9acf-2ed25e90af5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:24.323672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b71330-ec8d-4045-9acf-2ed25e90af5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:24.323672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b71330-ec8d-4045-9acf-2ed25e90af5b admin admin] PATH_INFO: `/v3/projects/73e5187bf3b74695a127aa9a0aa5b12d/users/9178a544490b4d879cd338104c9b46c0/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:24.359512 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b71330-ec8d-4045-9acf-2ed25e90af5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=73e5187bf3b74695a127aa9a0aa5b12d, user_id=9178a544490b4d879cd338104c9b46c0, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:24.366087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b71330-ec8d-4045-9acf-2ed25e90af5b admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:24.369330 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-56cc2b39-cc45-490f-96ba-b9c9800c74c4 None None] MFA Rules not processed for user `36b87ad6b4fb48f09393d9c561371fd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:24.372041 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-56cc2b39-cc45-490f-96ba-b9c9800c74c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:24.383326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:24.403712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 852/1688] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:24 2022] PUT /identity/v3/projects/73e5187bf3b74695a127aa9a0aa5b12d/users/9178a544490b4d879cd338104c9b46c0/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:24.403712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b71330-ec8d-4045-9acf-2ed25e90af5b admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:24.540388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b71330-ec8d-4045-9acf-2ed25e90af5b admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:24.541619 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 837/1689] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:24 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:24.549264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d0c56-f868-4730-a99b-cfbbd1c3ee8a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:24.549781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d0c56-f868-4730-a99b-cfbbd1c3ee8a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:24.550123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d0c56-f868-4730-a99b-cfbbd1c3ee8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:24.551140 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef80c3-cb56-4043-86ef-2d50e5716209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:24.552087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef80c3-cb56-4043-86ef-2d50e5716209 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:24.552477 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef80c3-cb56-4043-86ef-2d50e5716209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:24.552857 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef80c3-cb56-4043-86ef-2d50e5716209 admin admin] PATH_INFO: `/v3/projects/56768634acf2480dab17ed84719f3573/users/7a308a7cdbc243c7a643dd25f22a18df/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:24.554999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef80c3-cb56-4043-86ef-2d50e5716209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56768634acf2480dab17ed84719f3573, user_id=7a308a7cdbc243c7a643dd25f22a18df, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:24.563716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef80c3-cb56-4043-86ef-2d50e5716209 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:24.592480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 853/1690] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:24 2022] PUT /identity/v3/projects/56768634acf2480dab17ed84719f3573/users/7a308a7cdbc243c7a643dd25f22a18df/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:24.603641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd9675-fd85-4ba4-8ec5-936bca28d434 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:24.604082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd9675-fd85-4ba4-8ec5-936bca28d434 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:24.604465 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd9675-fd85-4ba4-8ec5-936bca28d434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:24.642958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7f4d0c56-f868-4730-a99b-cfbbd1c3ee8a None None] MFA Rules not processed for user `9178a544490b4d879cd338104c9b46c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:24.644995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7f4d0c56-f868-4730-a99b-cfbbd1c3ee8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:24.674530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-13bd9675-fd85-4ba4-8ec5-936bca28d434 None None] MFA Rules not processed for user `7a308a7cdbc243c7a643dd25f22a18df`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:24.679848 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-13bd9675-fd85-4ba4-8ec5-936bca28d434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:24.813397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 838/1691] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:24.814762 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d0c56-f868-4730-a99b-cfbbd1c3ee8a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:24.920279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 854/1692] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:30:24.937965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd9675-fd85-4ba4-8ec5-936bca28d434 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:24.940328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-13bd9675-fd85-4ba4-8ec5-936bca28d434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:24.964420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:24.986597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d0c56-f868-4730-a99b-cfbbd1c3ee8a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:24.999247 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcbc6b-a971-4247-ac0f-344987ffb460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:24.999948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcbc6b-a971-4247-ac0f-344987ffb460 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:25.000927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcbc6b-a971-4247-ac0f-344987ffb460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:25.000927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcbc6b-a971-4247-ac0f-344987ffb460 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:25.001395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcbc6b-a971-4247-ac0f-344987ffb460 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:25.003498 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcbc6b-a971-4247-ac0f-344987ffb460 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:25.048213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd9675-fd85-4ba4-8ec5-936bca28d434 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:25.054330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c398b-4226-4f34-91cd-74c158dfeaf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:25.055268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c398b-4226-4f34-91cd-74c158dfeaf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:25.055695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c398b-4226-4f34-91cd-74c158dfeaf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:25.056171 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c398b-4226-4f34-91cd-74c158dfeaf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:25.062773 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 839/1693] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:24 2022] POST /identity/v3/projects => generated 381 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:30:25.064809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcbc6b-a971-4247-ac0f-344987ffb460 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:25.076965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c398b-4226-4f34-91cd-74c158dfeaf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:25.081496 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c398b-4226-4f34-91cd-74c158dfeaf5 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:25.098373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 855/1694] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:25.193440 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcbc6b-a971-4247-ac0f-344987ffb460 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:25.217466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfbd2b-2deb-4fab-88fb-30640192ed20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:25.218180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfbd2b-2deb-4fab-88fb-30640192ed20 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:25.218470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfbd2b-2deb-4fab-88fb-30640192ed20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:25.219420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfbd2b-2deb-4fab-88fb-30640192ed20 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:25.219710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfbd2b-2deb-4fab-88fb-30640192ed20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:25.231294 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfbd2b-2deb-4fab-88fb-30640192ed20 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:25.274524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 840/1695] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:25 2022] POST /identity/v3/users => generated 297 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:30:25.279312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c398b-4226-4f34-91cd-74c158dfeaf5 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:25.297554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c398b-4226-4f34-91cd-74c158dfeaf5 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:25.337285 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f99176-f546-4bf3-b3a5-b4fb965f3c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:25.343065 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f99176-f546-4bf3-b3a5-b4fb965f3c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:25.344522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f99176-f546-4bf3-b3a5-b4fb965f3c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:25.344968 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f99176-f546-4bf3-b3a5-b4fb965f3c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:25.346416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f99176-f546-4bf3-b3a5-b4fb965f3c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:25.349852 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f99176-f546-4bf3-b3a5-b4fb965f3c2c admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:25.383606 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfbd2b-2deb-4fab-88fb-30640192ed20 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:25.384167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 856/1696] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:25.400662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfbd2b-2deb-4fab-88fb-30640192ed20 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:25.417318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd5261-c9e3-4ea6-8bab-ce12b24d5443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:25.417318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd5261-c9e3-4ea6-8bab-ce12b24d5443 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:25.417318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd5261-c9e3-4ea6-8bab-ce12b24d5443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:25.417318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd5261-c9e3-4ea6-8bab-ce12b24d5443 admin admin] PATH_INFO: `/v3/projects/980b2412c0e34520bb0db01eb8f172e4/users/05c1e44e62484930b9cef41213d0d20b/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:25.480133 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd5261-c9e3-4ea6-8bab-ce12b24d5443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=980b2412c0e34520bb0db01eb8f172e4, user_id=05c1e44e62484930b9cef41213d0d20b, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:25.492199 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd5261-c9e3-4ea6-8bab-ce12b24d5443 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:25.526265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 841/1697] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:25 2022] PUT /identity/v3/projects/980b2412c0e34520bb0db01eb8f172e4/users/05c1e44e62484930b9cef41213d0d20b/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:30:25.529764 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f99176-f546-4bf3-b3a5-b4fb965f3c2c admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:25.549528 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f99176-f546-4bf3-b3a5-b4fb965f3c2c admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:25.572119 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:25.680016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4a13d-04ac-45e8-aa59-4473cfd0c18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:25.680894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4a13d-04ac-45e8-aa59-4473cfd0c18b admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:25.682192 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4a13d-04ac-45e8-aa59-4473cfd0c18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:25.682485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4a13d-04ac-45e8-aa59-4473cfd0c18b admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:25.684433 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4a13d-04ac-45e8-aa59-4473cfd0c18b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:25.688639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4a13d-04ac-45e8-aa59-4473cfd0c18b admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:25.739725 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 857/1698] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:25.739992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd5261-c9e3-4ea6-8bab-ce12b24d5443 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:25.758623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd5261-c9e3-4ea6-8bab-ce12b24d5443 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:25.766873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e869c4f-2840-4a68-8f7b-a0f28bed7f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:25.766873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869c4f-2840-4a68-8f7b-a0f28bed7f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:25.773701 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869c4f-2840-4a68-8f7b-a0f28bed7f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:25.773853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869c4f-2840-4a68-8f7b-a0f28bed7f0e admin admin] PATH_INFO: `/v3/projects/980b2412c0e34520bb0db01eb8f172e4/users/05c1e44e62484930b9cef41213d0d20b/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:25.781452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e869c4f-2840-4a68-8f7b-a0f28bed7f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=980b2412c0e34520bb0db01eb8f172e4, user_id=05c1e44e62484930b9cef41213d0d20b, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:25.784184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e869c4f-2840-4a68-8f7b-a0f28bed7f0e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:25.820552 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 842/1699] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:25 2022] PUT /identity/v3/projects/980b2412c0e34520bb0db01eb8f172e4/users/05c1e44e62484930b9cef41213d0d20b/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:30:25.824857 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307b8cc-a658-402a-b4b5-0801873b01cb None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:25.825669 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307b8cc-a658-402a-b4b5-0801873b01cb None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:25.825963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307b8cc-a658-402a-b4b5-0801873b01cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:25.910848 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-5307b8cc-a658-402a-b4b5-0801873b01cb None None] MFA Rules not processed for user `05c1e44e62484930b9cef41213d0d20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:25.912656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5307b8cc-a658-402a-b4b5-0801873b01cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:26.027919 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e869c4f-2840-4a68-8f7b-a0f28bed7f0e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:26.167325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 858/1700] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:26.183105 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307b8cc-a658-402a-b4b5-0801873b01cb None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:26.184700 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5307b8cc-a658-402a-b4b5-0801873b01cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:26.246076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e869c4f-2840-4a68-8f7b-a0f28bed7f0e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:26.261619 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f437565-29e9-46b9-83e9-c682d0c8d0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:26.262522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f437565-29e9-46b9-83e9-c682d0c8d0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:26.262802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f437565-29e9-46b9-83e9-c682d0c8d0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:26.263077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f437565-29e9-46b9-83e9-c682d0c8d0b0 admin admin] PATH_INFO: `/v3/users/9f67b60fe38b4d86be9ad6a394336e9b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:26.268260 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f437565-29e9-46b9-83e9-c682d0c8d0b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f67b60fe38b4d86be9ad6a394336e9b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:26.271930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f437565-29e9-46b9-83e9-c682d0c8d0b0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:26.328416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307b8cc-a658-402a-b4b5-0801873b01cb None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:26.336602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba5af1-55c4-45cd-a512-41014afc4993 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:26.339887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba5af1-55c4-45cd-a512-41014afc4993 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:26.340488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba5af1-55c4-45cd-a512-41014afc4993 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:26.340828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba5af1-55c4-45cd-a512-41014afc4993 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:26.362275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba5af1-55c4-45cd-a512-41014afc4993 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:26.364835 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba5af1-55c4-45cd-a512-41014afc4993 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:26.366520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:26.399683 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 859/1701] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:26.400804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1f437565-29e9-46b9-83e9-c682d0c8d0b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f67b60fe38b4d86be9ad6a394336e9b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:26.403533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba5af1-55c4-45cd-a512-41014afc4993 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:26.410721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-71ba5af1-55c4-45cd-a512-41014afc4993 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:26.437832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1f437565-29e9-46b9-83e9-c682d0c8d0b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f67b60fe38b4d86be9ad6a394336e9b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:26.438605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1f437565-29e9-46b9-83e9-c682d0c8d0b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f67b60fe38b4d86be9ad6a394336e9b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:26.439240 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1f437565-29e9-46b9-83e9-c682d0c8d0b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f67b60fe38b4d86be9ad6a394336e9b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:26.455682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1f437565-29e9-46b9-83e9-c682d0c8d0b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f67b60fe38b4d86be9ad6a394336e9b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:26.483681 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba5af1-55c4-45cd-a512-41014afc4993 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:26.487704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 843/1702] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:26 2022] DELETE /identity/v3/users/9f67b60fe38b4d86be9ad6a394336e9b => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:26.489360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85997592-8594-46a2-b388-478783fd8592 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:26.489653 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85997592-8594-46a2-b388-478783fd8592 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:26.490870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85997592-8594-46a2-b388-478783fd8592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:26.543633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-85997592-8594-46a2-b388-478783fd8592 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:26.549746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-85997592-8594-46a2-b388-478783fd8592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:26.600014 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:26.687959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ecb24-009e-4593-8550-e790142adb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:26.696787 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ecb24-009e-4593-8550-e790142adb06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:26.699354 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ecb24-009e-4593-8550-e790142adb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:26.699781 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ecb24-009e-4593-8550-e790142adb06 admin admin] PATH_INFO: `/v3/projects/52837239ebfd465ebaccc583f4f1b1ac` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:26.703632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ecb24-009e-4593-8550-e790142adb06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52837239ebfd465ebaccc583f4f1b1ac)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:26.706303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ecb24-009e-4593-8550-e790142adb06 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:26.789169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 844/1703] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:26.790054 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85997592-8594-46a2-b388-478783fd8592 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:26.799017 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d40ecb24-009e-4593-8550-e790142adb06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52837239ebfd465ebaccc583f4f1b1ac', 'request_id': 'req-d40ecb24-009e-4593-8550-e790142adb06'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:26.803647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-85997592-8594-46a2-b388-478783fd8592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:26.808973 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d40ecb24-009e-4593-8550-e790142adb06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52837239ebfd465ebaccc583f4f1b1ac', 'request_id': 'req-d40ecb24-009e-4593-8550-e790142adb06'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:26.821022 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d40ecb24-009e-4593-8550-e790142adb06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52837239ebfd465ebaccc583f4f1b1ac', 'request_id': 'req-d40ecb24-009e-4593-8550-e790142adb06'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:26.841505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d40ecb24-009e-4593-8550-e790142adb06 admin admin] The token cache is being invalidate because project 52837239ebfd465ebaccc583f4f1b1ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:26.842140 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d40ecb24-009e-4593-8550-e790142adb06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:26.843179 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d40ecb24-009e-4593-8550-e790142adb06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:26.845911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 860/1704] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:26 2022] DELETE /identity/v3/projects/52837239ebfd465ebaccc583f4f1b1ac => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:26.851156 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739de821-1e40-46aa-9d5e-8996a60cdc27 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:26.851820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739de821-1e40-46aa-9d5e-8996a60cdc27 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:26.852329 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739de821-1e40-46aa-9d5e-8996a60cdc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:26.907977 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-739de821-1e40-46aa-9d5e-8996a60cdc27 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:26.910049 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-739de821-1e40-46aa-9d5e-8996a60cdc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:27.007684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 861/1705] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:27.007684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739de821-1e40-46aa-9d5e-8996a60cdc27 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:27.013936 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-739de821-1e40-46aa-9d5e-8996a60cdc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:27.014007 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85997592-8594-46a2-b388-478783fd8592 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:27.112396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739de821-1e40-46aa-9d5e-8996a60cdc27 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:27.117971 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcbe9d-eadf-4e2b-ad8a-67acc8aaf9f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:27.119399 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcbe9d-eadf-4e2b-ad8a-67acc8aaf9f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:27.120813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcbe9d-eadf-4e2b-ad8a-67acc8aaf9f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:27.121132 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcbe9d-eadf-4e2b-ad8a-67acc8aaf9f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:27.127027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-6bfcbe9d-eadf-4e2b-ad8a-67acc8aaf9f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:27.160534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfcbe9d-eadf-4e2b-ad8a-67acc8aaf9f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:27.166433 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfcbe9d-eadf-4e2b-ad8a-67acc8aaf9f2 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:27.195238 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 862/1706] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:27.198032 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cd889-c9e6-4913-ba72-e1ec8a0843f6 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:27.198330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cd889-c9e6-4913-ba72-e1ec8a0843f6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:27.198610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cd889-c9e6-4913-ba72-e1ec8a0843f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:27.237418 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-3a5cd889-c9e6-4913-ba72-e1ec8a0843f6 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:27.240328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3a5cd889-c9e6-4913-ba72-e1ec8a0843f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:27.260553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 863/1707] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:27.265747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cd889-c9e6-4913-ba72-e1ec8a0843f6 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:27.273857 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:27.282845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cd889-c9e6-4913-ba72-e1ec8a0843f6 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:27.286721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb36a1a-3e65-49be-8cd7-74fc2d2c4849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:27.287418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb36a1a-3e65-49be-8cd7-74fc2d2c4849 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:27.290812 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb36a1a-3e65-49be-8cd7-74fc2d2c4849 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:27.291453 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb36a1a-3e65-49be-8cd7-74fc2d2c4849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:27.296924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f957f-bb94-4517-a0d4-7e3a7a9b4bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:27.297739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f957f-bb94-4517-a0d4-7e3a7a9b4bfe admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:27.298062 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-eeb36a1a-3e65-49be-8cd7-74fc2d2c4849 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:27.298173 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f957f-bb94-4517-a0d4-7e3a7a9b4bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:27.298473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f957f-bb94-4517-a0d4-7e3a7a9b4bfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:27.299534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f957f-bb94-4517-a0d4-7e3a7a9b4bfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:27.300973 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f957f-bb94-4517-a0d4-7e3a7a9b4bfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:27.302851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f957f-bb94-4517-a0d4-7e3a7a9b4bfe admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:27.323201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 864/1708] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:30:27.324376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f957f-bb94-4517-a0d4-7e3a7a9b4bfe admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:27.336250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f957f-bb94-4517-a0d4-7e3a7a9b4bfe admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:27.345544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b01f2-f190-4891-b6ac-338f83bb853d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:27.346904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b01f2-f190-4891-b6ac-338f83bb853d admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:27.347332 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b01f2-f190-4891-b6ac-338f83bb853d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:27.347760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b01f2-f190-4891-b6ac-338f83bb853d admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:27.348665 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b01f2-f190-4891-b6ac-338f83bb853d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:27.350898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b01f2-f190-4891-b6ac-338f83bb853d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:27.358235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:27.402429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 865/1709] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:27 2022] POST /identity/v3/projects => generated 367 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:27.403384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b01f2-f190-4891-b6ac-338f83bb853d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:27.482048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb36a1a-3e65-49be-8cd7-74fc2d2c4849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:27.484809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb36a1a-3e65-49be-8cd7-74fc2d2c4849 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:27.499346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 845/1710] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:27.577286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b01f2-f190-4891-b6ac-338f83bb853d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:27.583782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc38ebe-0fbb-4b16-9d81-d72b22b224fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:27.584871 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc38ebe-0fbb-4b16-9d81-d72b22b224fc admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:27.585379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc38ebe-0fbb-4b16-9d81-d72b22b224fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:27.585801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc38ebe-0fbb-4b16-9d81-d72b22b224fc admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:27.586844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc38ebe-0fbb-4b16-9d81-d72b22b224fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:27.590158 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc38ebe-0fbb-4b16-9d81-d72b22b224fc admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:27.625213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 866/1711] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:27 2022] POST /identity/v3/users => generated 284 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:27.628988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb36a1a-3e65-49be-8cd7-74fc2d2c4849 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:27.652051 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb36a1a-3e65-49be-8cd7-74fc2d2c4849 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:27.662290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8d8d6-7cb8-4d39-af8a-172abe886e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:27.663184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8d8d6-7cb8-4d39-af8a-172abe886e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:27.664202 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8d8d6-7cb8-4d39-af8a-172abe886e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:27.664289 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8d8d6-7cb8-4d39-af8a-172abe886e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:27.666216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8d8d6-7cb8-4d39-af8a-172abe886e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:27.669125 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8d8d6-7cb8-4d39-af8a-172abe886e20 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:27.696480 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 846/1712] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:30:27.701931 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc38ebe-0fbb-4b16-9d81-d72b22b224fc admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:27.724929 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc38ebe-0fbb-4b16-9d81-d72b22b224fc admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:27.736500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b82986-2394-434d-87a4-43b94e5e35c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:27.737332 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b82986-2394-434d-87a4-43b94e5e35c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:27.737625 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b82986-2394-434d-87a4-43b94e5e35c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:27.737907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b82986-2394-434d-87a4-43b94e5e35c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:27.738778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b82986-2394-434d-87a4-43b94e5e35c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:27.742826 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b82986-2394-434d-87a4-43b94e5e35c6 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:27.761276 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 867/1713] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:27.764355 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8d8d6-7cb8-4d39-af8a-172abe886e20 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:27.784215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8d8d6-7cb8-4d39-af8a-172abe886e20 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:27.795520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9e275-1da5-4ac6-90b8-aa0f2b726ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:27.803659 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9e275-1da5-4ac6-90b8-aa0f2b726ae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:27.806959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9e275-1da5-4ac6-90b8-aa0f2b726ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:27.807809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9e275-1da5-4ac6-90b8-aa0f2b726ae2 admin admin] PATH_INFO: `/v3/projects/02f097a87fa6490a886516d83fe95c55/users/8d8ed9319c1740eb92b05a1453d5599e/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:27.844425 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1baa53-de1e-4f99-8f43-9b078d98e120 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:27.844877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1baa53-de1e-4f99-8f43-9b078d98e120 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:27.845826 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1baa53-de1e-4f99-8f43-9b078d98e120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:27.903318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9e275-1da5-4ac6-90b8-aa0f2b726ae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f097a87fa6490a886516d83fe95c55, user_id=8d8ed9319c1740eb92b05a1453d5599e, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:27.917289 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9e275-1da5-4ac6-90b8-aa0f2b726ae2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:27.968687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-fd1baa53-de1e-4f99-8f43-9b078d98e120 None None] MFA Rules not processed for user `064d7bbcf3684ccbafc5f3d1cb561d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:27.973869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 847/1714] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:27 2022] PUT /identity/v3/projects/02f097a87fa6490a886516d83fe95c55/users/8d8ed9319c1740eb92b05a1453d5599e/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:27.974046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-fd1baa53-de1e-4f99-8f43-9b078d98e120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:27.981443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9452ac8-df67-428a-ad43-d3bb4a8a0bdd None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:27.981733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9452ac8-df67-428a-ad43-d3bb4a8a0bdd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:27.982164 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9452ac8-df67-428a-ad43-d3bb4a8a0bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:28.083576 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f9452ac8-df67-428a-ad43-d3bb4a8a0bdd None None] MFA Rules not processed for user `8d8ed9319c1740eb92b05a1453d5599e`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:28.090660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f9452ac8-df67-428a-ad43-d3bb4a8a0bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:28.238150 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 868/1715] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:28.247657 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaa175-d296-4cb2-a7d4-b8dc33e83dcd None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:28.248334 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaa175-d296-4cb2-a7d4-b8dc33e83dcd None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:28.248757 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaa175-d296-4cb2-a7d4-b8dc33e83dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:28.307543 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-0ceaa175-d296-4cb2-a7d4-b8dc33e83dcd None None] MFA Rules not processed for user `064d7bbcf3684ccbafc5f3d1cb561d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:28.308744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0ceaa175-d296-4cb2-a7d4-b8dc33e83dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:28.332138 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 848/1716] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:28.339842 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9452ac8-df67-428a-ad43-d3bb4a8a0bdd None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:28.340036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 869/1717] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:28.352588 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaa175-d296-4cb2-a7d4-b8dc33e83dcd None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:28.358110 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0ceaa175-d296-4cb2-a7d4-b8dc33e83dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:28.468776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:28.555305 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9452ac8-df67-428a-ad43-d3bb4a8a0bdd None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:28.580724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb9153c-7597-4185-bbbf-774c0c3131b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:28.581569 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9153c-7597-4185-bbbf-774c0c3131b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:28.582029 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9153c-7597-4185-bbbf-774c0c3131b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:28.582451 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9153c-7597-4185-bbbf-774c0c3131b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:28.583397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb9153c-7597-4185-bbbf-774c0c3131b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:28.587092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb9153c-7597-4185-bbbf-774c0c3131b1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:28.598678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaa175-d296-4cb2-a7d4-b8dc33e83dcd None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:28.625707 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2f95d-e72a-496c-922a-2e17e626d578 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:28.626410 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2f95d-e72a-496c-922a-2e17e626d578 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:28.627217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2f95d-e72a-496c-922a-2e17e626d578 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:28.627530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2f95d-e72a-496c-922a-2e17e626d578 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:28.652417 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2f95d-e72a-496c-922a-2e17e626d578 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:28.654121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2f95d-e72a-496c-922a-2e17e626d578 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:28.656004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:28.673755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 849/1718] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:28 2022] POST /identity/v3/projects => generated 365 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:28.678351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb9153c-7597-4185-bbbf-774c0c3131b1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:28.680521 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 870/1719] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:28.680706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2f95d-e72a-496c-922a-2e17e626d578 service nova] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:28.683050 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8fb9153c-7597-4185-bbbf-774c0c3131b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:28.701757 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2f95d-e72a-496c-922a-2e17e626d578 service nova] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:28.821630 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb9153c-7597-4185-bbbf-774c0c3131b1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:28.835850 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd08e9-196d-4f4f-8d2a-be35e70ca47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:28.838173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd08e9-196d-4f4f-8d2a-be35e70ca47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:28.839684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd08e9-196d-4f4f-8d2a-be35e70ca47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:28.840004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd08e9-196d-4f4f-8d2a-be35e70ca47e admin admin] PATH_INFO: `/v3/projects/680475cbe61a44ba982cc0de3c0e301b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:28.843972 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9603360-8954-478c-aabd-f5fcc8f85cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:28.844144 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd08e9-196d-4f4f-8d2a-be35e70ca47e admin admin] RBAC: Authorizing `identity:delete_project(project_id=680475cbe61a44ba982cc0de3c0e301b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:28.844665 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9603360-8954-478c-aabd-f5fcc8f85cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:28.844954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9603360-8954-478c-aabd-f5fcc8f85cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:28.845305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9603360-8954-478c-aabd-f5fcc8f85cc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:28.846018 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9603360-8954-478c-aabd-f5fcc8f85cc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:28.846137 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd08e9-196d-4f4f-8d2a-be35e70ca47e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:28.847957 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9603360-8954-478c-aabd-f5fcc8f85cc4 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:28.888241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 871/1720] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:28 2022] POST /identity/v3/users => generated 289 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:28.892159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9603360-8954-478c-aabd-f5fcc8f85cc4 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:28.918071 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9603360-8954-478c-aabd-f5fcc8f85cc4 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:28.929841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaaa479-2383-4130-98d1-2b5a0ceadd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:28.930530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaa479-2383-4130-98d1-2b5a0ceadd11 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:28.930824 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaa479-2383-4130-98d1-2b5a0ceadd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:28.931107 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaa479-2383-4130-98d1-2b5a0ceadd11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:28.932980 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaaa479-2383-4130-98d1-2b5a0ceadd11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:28.934659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaaa479-2383-4130-98d1-2b5a0ceadd11 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:28.972480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 872/1721] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:30:28.973944 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a0cb6-9e35-4698-88ec-e686f6d7d916 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:28.974566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a0cb6-9e35-4698-88ec-e686f6d7d916 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:28.977177 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a0cb6-9e35-4698-88ec-e686f6d7d916 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:28.979264 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 873/1722] 10.222.0.42 () {54 vars in 921 bytes} [Tue Jul 19 15:30:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:30:28.984548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a0cb6-9e35-4698-88ec-e686f6d7d916 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:29.021101 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ebcd08e9-196d-4f4f-8d2a-be35e70ca47e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '680475cbe61a44ba982cc0de3c0e301b', 'request_id': 'req-ebcd08e9-196d-4f4f-8d2a-be35e70ca47e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:29.043736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ebcd08e9-196d-4f4f-8d2a-be35e70ca47e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '680475cbe61a44ba982cc0de3c0e301b', 'request_id': 'req-ebcd08e9-196d-4f4f-8d2a-be35e70ca47e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:29.071826 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ebcd08e9-196d-4f4f-8d2a-be35e70ca47e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '680475cbe61a44ba982cc0de3c0e301b', 'request_id': 'req-ebcd08e9-196d-4f4f-8d2a-be35e70ca47e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:29.113661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ebcd08e9-196d-4f4f-8d2a-be35e70ca47e admin admin] The token cache is being invalidate because project 680475cbe61a44ba982cc0de3c0e301b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:29.114745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ebcd08e9-196d-4f4f-8d2a-be35e70ca47e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:29.116725 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ebcd08e9-196d-4f4f-8d2a-be35e70ca47e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:29.130274 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 850/1723] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:28 2022] DELETE /identity/v3/projects/680475cbe61a44ba982cc0de3c0e301b => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:29.133697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67e0cc-0e3e-49a7-aca4-415962b98561 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:29.135912 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67e0cc-0e3e-49a7-aca4-415962b98561 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:29.137552 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67e0cc-0e3e-49a7-aca4-415962b98561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:29.183426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ff67e0cc-0e3e-49a7-aca4-415962b98561 None None] MFA Rules not processed for user `6002f32936944b0db6fd04f3851836db`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:29.185507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ff67e0cc-0e3e-49a7-aca4-415962b98561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:29.231617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a0cb6-9e35-4698-88ec-e686f6d7d916 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:29.240875 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b530cb0-1663-48a3-9540-9f34bf7481fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:29.241895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b530cb0-1663-48a3-9540-9f34bf7481fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:29.242152 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b530cb0-1663-48a3-9540-9f34bf7481fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:29.242432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b530cb0-1663-48a3-9540-9f34bf7481fb admin admin] PATH_INFO: `/v3/projects/db1189b9007e4b42976625151c757d3d/users/a7c6ee18f08a41de8489b09dccfc73f6/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:29.276615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b530cb0-1663-48a3-9540-9f34bf7481fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1189b9007e4b42976625151c757d3d, user_id=a7c6ee18f08a41de8489b09dccfc73f6, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:29.281713 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b530cb0-1663-48a3-9540-9f34bf7481fb admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:29.313843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 874/1724] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:28 2022] PUT /identity/v3/projects/db1189b9007e4b42976625151c757d3d/users/a7c6ee18f08a41de8489b09dccfc73f6/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:29.328370 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b530cb0-1663-48a3-9540-9f34bf7481fb admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:29.336304 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5b530cb0-1663-48a3-9540-9f34bf7481fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:29.379014 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:29.467259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 851/1725] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Jul 19 15:30:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:29.469615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67e0cc-0e3e-49a7-aca4-415962b98561 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:29.476167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ff67e0cc-0e3e-49a7-aca4-415962b98561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:29.496217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b530cb0-1663-48a3-9540-9f34bf7481fb admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:29.503103 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d337667-043d-4ece-928d-a14fc301702c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:29.504584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d337667-043d-4ece-928d-a14fc301702c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:29.505179 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d337667-043d-4ece-928d-a14fc301702c admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:29.505659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d337667-043d-4ece-928d-a14fc301702c admin admin] PATH_INFO: `/v3/users/7a308a7cdbc243c7a643dd25f22a18df` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:29.507052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d337667-043d-4ece-928d-a14fc301702c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a308a7cdbc243c7a643dd25f22a18df)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:29.509186 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d337667-043d-4ece-928d-a14fc301702c admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:29.536257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67e0cc-0e3e-49a7-aca4-415962b98561 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:29.562870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dea240-021f-4a97-869f-6edf10363b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:29.563604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dea240-021f-4a97-869f-6edf10363b01 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:29.563886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dea240-021f-4a97-869f-6edf10363b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:29.564158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dea240-021f-4a97-869f-6edf10363b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:29.565597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dea240-021f-4a97-869f-6edf10363b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:29.567840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dea240-021f-4a97-869f-6edf10363b01 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:29.585184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 852/1726] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:29.586392 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dea240-021f-4a97-869f-6edf10363b01 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:29.598589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-4d337667-043d-4ece-928d-a14fc301702c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a308a7cdbc243c7a643dd25f22a18df'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:29.600605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dea240-021f-4a97-869f-6edf10363b01 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:29.609674 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f210a0d-5176-457a-9c3b-b79030d7f133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:29.610961 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f210a0d-5176-457a-9c3b-b79030d7f133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:29.611283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f210a0d-5176-457a-9c3b-b79030d7f133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:29.611567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f210a0d-5176-457a-9c3b-b79030d7f133 admin admin] PATH_INFO: `/v3/projects/db1189b9007e4b42976625151c757d3d/users/a7c6ee18f08a41de8489b09dccfc73f6/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:29.613839 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f210a0d-5176-457a-9c3b-b79030d7f133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1189b9007e4b42976625151c757d3d, user_id=a7c6ee18f08a41de8489b09dccfc73f6, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:29.616743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f210a0d-5176-457a-9c3b-b79030d7f133 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:29.621174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-4d337667-043d-4ece-928d-a14fc301702c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a308a7cdbc243c7a643dd25f22a18df'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:29.622169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-4d337667-043d-4ece-928d-a14fc301702c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a308a7cdbc243c7a643dd25f22a18df'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:29.632636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-4d337667-043d-4ece-928d-a14fc301702c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a308a7cdbc243c7a643dd25f22a18df'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:29.634891 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-4d337667-043d-4ece-928d-a14fc301702c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a308a7cdbc243c7a643dd25f22a18df'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:29.643855 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 853/1727] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:29 2022] PUT /identity/v3/projects/db1189b9007e4b42976625151c757d3d/users/a7c6ee18f08a41de8489b09dccfc73f6/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:29.645791 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e274f7e-dd91-44e7-b63a-b537ecf5dc02 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:29.646127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e274f7e-dd91-44e7-b63a-b537ecf5dc02 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:29.646403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e274f7e-dd91-44e7-b63a-b537ecf5dc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:29.658771 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 875/1728] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:29 2022] DELETE /identity/v3/users/7a308a7cdbc243c7a643dd25f22a18df => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:29.661650 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:29.663635 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6ce6e-c68d-4470-b7e2-f74f85adb5b9 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:29.668162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6ce6e-c68d-4470-b7e2-f74f85adb5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:29.668780 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6ce6e-c68d-4470-b7e2-f74f85adb5b9 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:29.672298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 876/1729] 10.222.0.42 () {54 vars in 961 bytes} [Tue Jul 19 15:30:29 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:30:29.678080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086dfb5-f888-4914-a206-d4da0b767623 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:29.678450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086dfb5-f888-4914-a206-d4da0b767623 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:29.678796 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086dfb5-f888-4914-a206-d4da0b767623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:29.709394 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3e274f7e-dd91-44e7-b63a-b537ecf5dc02 None None] MFA Rules not processed for user `a7c6ee18f08a41de8489b09dccfc73f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:29.710869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3e274f7e-dd91-44e7-b63a-b537ecf5dc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:29.741160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-e086dfb5-f888-4914-a206-d4da0b767623 None None] MFA Rules not processed for user `6002f32936944b0db6fd04f3851836db`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:29.742906 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e086dfb5-f888-4914-a206-d4da0b767623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:29.907076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 854/1730] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:29 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:29.938802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e274f7e-dd91-44e7-b63a-b537ecf5dc02 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:29.940627 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3e274f7e-dd91-44e7-b63a-b537ecf5dc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:29.961390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 877/1731] 10.222.0.42 () {58 vars in 1058 bytes} [Tue Jul 19 15:30:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:30:29.967711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086dfb5-f888-4914-a206-d4da0b767623 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:29.996919 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086dfb5-f888-4914-a206-d4da0b767623 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:30.009422 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31849eb-1b38-4dec-990c-de475a998f3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:30.009743 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31849eb-1b38-4dec-990c-de475a998f3a service placement] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:30.010084 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31849eb-1b38-4dec-990c-de475a998f3a service placement] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:30.010363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31849eb-1b38-4dec-990c-de475a998f3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:30.016065 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c31849eb-1b38-4dec-990c-de475a998f3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:30.055248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31849eb-1b38-4dec-990c-de475a998f3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:30.057687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31849eb-1b38-4dec-990c-de475a998f3a service placement] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:30.087494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e274f7e-dd91-44e7-b63a-b537ecf5dc02 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:30.091981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a999b-b7b5-41b1-a253-a6441c18bae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:30.092656 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a999b-b7b5-41b1-a253-a6441c18bae7 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:30.093580 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a999b-b7b5-41b1-a253-a6441c18bae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:30.093580 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a999b-b7b5-41b1-a253-a6441c18bae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:30.099303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 878/1732] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:30.106418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a999b-b7b5-41b1-a253-a6441c18bae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:30.108007 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a999b-b7b5-41b1-a253-a6441c18bae7 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:30.121073 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 855/1733] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:29 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:30:30.262305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31849eb-1b38-4dec-990c-de475a998f3a service placement] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:30.275016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31849eb-1b38-4dec-990c-de475a998f3a service placement] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:30.287941 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef6c56-a4e1-4bd0-83a1-bacd69ad6bfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:30.288787 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef6c56-a4e1-4bd0-83a1-bacd69ad6bfc service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:30.290732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef6c56-a4e1-4bd0-83a1-bacd69ad6bfc service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:30.291297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef6c56-a4e1-4bd0-83a1-bacd69ad6bfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:30.296249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2bef6c56-a4e1-4bd0-83a1-bacd69ad6bfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:30.335581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:30.436200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef6c56-a4e1-4bd0-83a1-bacd69ad6bfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:30.437815 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef6c56-a4e1-4bd0-83a1-bacd69ad6bfc service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:30.448853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 879/1734] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:30.684405 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef57f76-6920-4d90-b617-2f48ceef1318 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:30.684699 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef57f76-6920-4d90-b617-2f48ceef1318 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:30.684983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef57f76-6920-4d90-b617-2f48ceef1318 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:30.686423 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 856/1735] 10.222.0.42 () {54 vars in 912 bytes} [Tue Jul 19 15:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:30:30.692323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535f9c2-dcf3-4862-adca-e541bc376609 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:30.692613 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535f9c2-dcf3-4862-adca-e541bc376609 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:30.692897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535f9c2-dcf3-4862-adca-e541bc376609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:30.861189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-2535f9c2-dcf3-4862-adca-e541bc376609 None None] MFA Rules not processed for user `369bdaded0534142b03ad3cf43a70881`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:30.865079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2535f9c2-dcf3-4862-adca-e541bc376609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:31.017459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 880/1736] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:30:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:30:31.268722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef57f76-6920-4d90-b617-2f48ceef1318 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:31.270609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4ef57f76-6920-4d90-b617-2f48ceef1318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:31.308590 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef57f76-6920-4d90-b617-2f48ceef1318 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:31.337856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09958d-7260-4d2d-a58d-1e37da478ad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:31.338249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09958d-7260-4d2d-a58d-1e37da478ad2 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:31.338541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09958d-7260-4d2d-a58d-1e37da478ad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:31.338823 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09958d-7260-4d2d-a58d-1e37da478ad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:31.378000 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09958d-7260-4d2d-a58d-1e37da478ad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:31.380067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09958d-7260-4d2d-a58d-1e37da478ad2 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:31.435903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 857/1737] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:31.486716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535f9c2-dcf3-4862-adca-e541bc376609 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:31.489464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2535f9c2-dcf3-4862-adca-e541bc376609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:31.674019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535f9c2-dcf3-4862-adca-e541bc376609 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:31.704356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3168c2f-9148-4e8f-941d-3c3c082c3271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:31.705865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3168c2f-9148-4e8f-941d-3c3c082c3271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:31.706163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3168c2f-9148-4e8f-941d-3c3c082c3271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:31.707063 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3168c2f-9148-4e8f-941d-3c3c082c3271 admin admin] PATH_INFO: `/v3/projects/56768634acf2480dab17ed84719f3573` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:31.717891 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3168c2f-9148-4e8f-941d-3c3c082c3271 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56768634acf2480dab17ed84719f3573)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:31.727229 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3168c2f-9148-4e8f-941d-3c3c082c3271 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:31.887888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09958d-7260-4d2d-a58d-1e37da478ad2 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:31.889673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6a09958d-7260-4d2d-a58d-1e37da478ad2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:31.897963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a3168c2f-9148-4e8f-941d-3c3c082c3271 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56768634acf2480dab17ed84719f3573', 'request_id': 'req-a3168c2f-9148-4e8f-941d-3c3c082c3271'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:31.918398 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:31.928190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a3168c2f-9148-4e8f-941d-3c3c082c3271 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56768634acf2480dab17ed84719f3573', 'request_id': 'req-a3168c2f-9148-4e8f-941d-3c3c082c3271'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:31.938882 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09958d-7260-4d2d-a58d-1e37da478ad2 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:31.954071 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a3168c2f-9148-4e8f-941d-3c3c082c3271 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56768634acf2480dab17ed84719f3573', 'request_id': 'req-a3168c2f-9148-4e8f-941d-3c3c082c3271'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:31.990791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a3168c2f-9148-4e8f-941d-3c3c082c3271 admin admin] The token cache is being invalidate because project 56768634acf2480dab17ed84719f3573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:31.994242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a3168c2f-9148-4e8f-941d-3c3c082c3271 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:31.998036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a3168c2f-9148-4e8f-941d-3c3c082c3271 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:32.003443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 881/1738] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:31 2022] DELETE /identity/v3/projects/56768634acf2480dab17ed84719f3573 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:32.005681 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada08578-4b5d-45c5-87ba-36f4197de129 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:32.006008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada08578-4b5d-45c5-87ba-36f4197de129 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:32.006260 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada08578-4b5d-45c5-87ba-36f4197de129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:32.060309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ada08578-4b5d-45c5-87ba-36f4197de129 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:32.061621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ada08578-4b5d-45c5-87ba-36f4197de129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:32.156196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6fb2f-eb3d-456c-8028-6883d84257d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:32.157715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6fb2f-eb3d-456c-8028-6883d84257d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:32.157999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6fb2f-eb3d-456c-8028-6883d84257d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:32.158310 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6fb2f-eb3d-456c-8028-6883d84257d1 admin admin] PATH_INFO: `/v3/users/46cd0aa86f274786b53c492a72c199cd` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:32.159511 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6fb2f-eb3d-456c-8028-6883d84257d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46cd0aa86f274786b53c492a72c199cd)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:32.161777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6fb2f-eb3d-456c-8028-6883d84257d1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:32.200057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:32.232273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 882/1739] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:32.233762 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6442167-dcd4-4721-b4f6-0436bf712fde None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:32.236040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6442167-dcd4-4721-b4f6-0436bf712fde None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:32.236313 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6442167-dcd4-4721-b4f6-0436bf712fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:32.292352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-91b6fb2f-eb3d-456c-8028-6883d84257d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46cd0aa86f274786b53c492a72c199cd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:32.329394 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-91b6fb2f-eb3d-456c-8028-6883d84257d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46cd0aa86f274786b53c492a72c199cd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:32.331290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-91b6fb2f-eb3d-456c-8028-6883d84257d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46cd0aa86f274786b53c492a72c199cd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:32.333284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-91b6fb2f-eb3d-456c-8028-6883d84257d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46cd0aa86f274786b53c492a72c199cd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:32.361942 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-91b6fb2f-eb3d-456c-8028-6883d84257d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46cd0aa86f274786b53c492a72c199cd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:32.386949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-f6442167-dcd4-4721-b4f6-0436bf712fde None None] MFA Rules not processed for user `ddb5e1dec1904ae183e131ba95ed1ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:32.397360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f6442167-dcd4-4721-b4f6-0436bf712fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:32.407560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 858/1740] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:31 2022] DELETE /identity/v3/users/46cd0aa86f274786b53c492a72c199cd => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:30:32.412440 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040419e0-1804-41fb-8e87-3191439ef77a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:32.412829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040419e0-1804-41fb-8e87-3191439ef77a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:32.413377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040419e0-1804-41fb-8e87-3191439ef77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:32.461527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-040419e0-1804-41fb-8e87-3191439ef77a None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:32.465256 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-040419e0-1804-41fb-8e87-3191439ef77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:32.617925 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 883/1741] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:32.622591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a45e96-5b8d-493c-a30a-9f61a6cc38ba None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:32.622836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a45e96-5b8d-493c-a30a-9f61a6cc38ba None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:32.623958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a45e96-5b8d-493c-a30a-9f61a6cc38ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:32.666879 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 859/1742] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:32.672220 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040419e0-1804-41fb-8e87-3191439ef77a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:32.676297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-040419e0-1804-41fb-8e87-3191439ef77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:32.757922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-93a45e96-5b8d-493c-a30a-9f61a6cc38ba None None] MFA Rules not processed for user `5507329a6bb84cfbbdac99b85f346ab8`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:32.761889 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-93a45e96-5b8d-493c-a30a-9f61a6cc38ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:33.146977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040419e0-1804-41fb-8e87-3191439ef77a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:33.164804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd11210-4c20-4945-b681-55cd848deda9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:33.166399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd11210-4c20-4945-b681-55cd848deda9 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:33.169405 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd11210-4c20-4945-b681-55cd848deda9 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:33.174756 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd11210-4c20-4945-b681-55cd848deda9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:33.211373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 884/1743] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 592 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:33.217641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a45e96-5b8d-493c-a30a-9f61a6cc38ba None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:33.239686 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:33.271973 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a45e96-5b8d-493c-a30a-9f61a6cc38ba None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:33.280518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ad3f0-a632-4602-89b4-f58496613e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:33.281575 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ad3f0-a632-4602-89b4-f58496613e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:33.282053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ad3f0-a632-4602-89b4-f58496613e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:33.282669 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ad3f0-a632-4602-89b4-f58496613e90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:33.283728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ad3f0-a632-4602-89b4-f58496613e90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:33.284333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ad3f0-a632-4602-89b4-f58496613e90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:33.286578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ad3f0-a632-4602-89b4-f58496613e90 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:33.297764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:33.307807 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 885/1744] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:30:33.309083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ad3f0-a632-4602-89b4-f58496613e90 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:33.317728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b74ad3f0-a632-4602-89b4-f58496613e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:33.346345 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ad3f0-a632-4602-89b4-f58496613e90 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:33.352495 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d87412-9604-40e4-83da-8137b07ed4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:33.354312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d87412-9604-40e4-83da-8137b07ed4fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:33.355002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d87412-9604-40e4-83da-8137b07ed4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:33.355471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d87412-9604-40e4-83da-8137b07ed4fe admin admin] PATH_INFO: `/v3/projects/3eac7356ebf340139bbc28015ec21472` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:33.357084 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d87412-9604-40e4-83da-8137b07ed4fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eac7356ebf340139bbc28015ec21472)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:33.359785 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d87412-9604-40e4-83da-8137b07ed4fe admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:33.379722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd11210-4c20-4945-b681-55cd848deda9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:33.382332 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd11210-4c20-4945-b681-55cd848deda9 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:33.399093 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 860/1745] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 735 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:33.411564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd11210-4c20-4945-b681-55cd848deda9 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:33.411564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4dd11210-4c20-4945-b681-55cd848deda9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:33.471870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-58d87412-9604-40e4-83da-8137b07ed4fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eac7356ebf340139bbc28015ec21472', 'request_id': 'req-58d87412-9604-40e4-83da-8137b07ed4fe'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:33.479192 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-58d87412-9604-40e4-83da-8137b07ed4fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eac7356ebf340139bbc28015ec21472', 'request_id': 'req-58d87412-9604-40e4-83da-8137b07ed4fe'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:33.491568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-58d87412-9604-40e4-83da-8137b07ed4fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eac7356ebf340139bbc28015ec21472', 'request_id': 'req-58d87412-9604-40e4-83da-8137b07ed4fe'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:33.509229 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-58d87412-9604-40e4-83da-8137b07ed4fe admin admin] The token cache is being invalidate because project 3eac7356ebf340139bbc28015ec21472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:33.510047 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-58d87412-9604-40e4-83da-8137b07ed4fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:33.513282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-58d87412-9604-40e4-83da-8137b07ed4fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:33.516574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 886/1746] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:33 2022] DELETE /identity/v3/projects/3eac7356ebf340139bbc28015ec21472 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:33.518555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d87412-9604-40e4-83da-8137b07ed4fe admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:33.520439 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-58d87412-9604-40e4-83da-8137b07ed4fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:33.539268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd11210-4c20-4945-b681-55cd848deda9 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:33.638154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d87412-9604-40e4-83da-8137b07ed4fe admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:33.644031 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd63911-b23b-4d9e-b4d6-4d37c41d3536 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:33.644923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd63911-b23b-4d9e-b4d6-4d37c41d3536 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:33.645289 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd63911-b23b-4d9e-b4d6-4d37c41d3536 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:33.645575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd63911-b23b-4d9e-b4d6-4d37c41d3536 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:33.648691 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7fd63911-b23b-4d9e-b4d6-4d37c41d3536 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:33.652336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fa18e-7210-4c4c-b79a-7ca6405876b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:33.657354 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa18e-7210-4c4c-b79a-7ca6405876b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:33.657681 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa18e-7210-4c4c-b79a-7ca6405876b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:33.657938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa18e-7210-4c4c-b79a-7ca6405876b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:33.659850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fa18e-7210-4c4c-b79a-7ca6405876b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:33.665033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fa18e-7210-4c4c-b79a-7ca6405876b7 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:33.681677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd63911-b23b-4d9e-b4d6-4d37c41d3536 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:33.683558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd63911-b23b-4d9e-b4d6-4d37c41d3536 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:33.706222 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 861/1747] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:33.710966 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd63911-b23b-4d9e-b4d6-4d37c41d3536 service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:33.717302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7fd63911-b23b-4d9e-b4d6-4d37c41d3536 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:33.729990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 887/1748] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:33 2022] POST /identity/v3/projects => generated 377 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:33.731529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6b80e-2bc7-4949-8380-93dfea515aef None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:33.735039 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6b80e-2bc7-4949-8380-93dfea515aef None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:33.735331 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6b80e-2bc7-4949-8380-93dfea515aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:33.785227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-65b6b80e-2bc7-4949-8380-93dfea515aef None None] MFA Rules not processed for user `51eb5efc37ca4942831bc4d046f54da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:33.786539 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-65b6b80e-2bc7-4949-8380-93dfea515aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:33.881903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:33.902935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:33.908793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd63911-b23b-4d9e-b4d6-4d37c41d3536 service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:33.909030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 888/1749] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:33.909990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6b80e-2bc7-4949-8380-93dfea515aef None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:33.913610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-65b6b80e-2bc7-4949-8380-93dfea515aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:34.095288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a167080-0b55-4765-bd90-8f8e54e60b8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:34.096227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a167080-0b55-4765-bd90-8f8e54e60b8a service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:34.097469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a167080-0b55-4765-bd90-8f8e54e60b8a service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:34.102081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a167080-0b55-4765-bd90-8f8e54e60b8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:34.107470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2a167080-0b55-4765-bd90-8f8e54e60b8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:34.125839 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6b80e-2bc7-4949-8380-93dfea515aef None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:34.131290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d00c0-547d-486b-9297-ce1b1086363d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:34.132552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d00c0-547d-486b-9297-ce1b1086363d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:34.133326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d00c0-547d-486b-9297-ce1b1086363d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:34.133822 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d00c0-547d-486b-9297-ce1b1086363d admin admin] PATH_INFO: `/v3/users/36b87ad6b4fb48f09393d9c561371fd1` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:34.135842 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d00c0-547d-486b-9297-ce1b1086363d admin admin] RBAC: Authorizing `identity:delete_user(user_id=36b87ad6b4fb48f09393d9c561371fd1)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:34.138401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d00c0-547d-486b-9297-ce1b1086363d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:34.285361 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a167080-0b55-4765-bd90-8f8e54e60b8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:34.288433 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d92d00c0-547d-486b-9297-ce1b1086363d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36b87ad6b4fb48f09393d9c561371fd1'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:34.289803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a167080-0b55-4765-bd90-8f8e54e60b8a service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:34.305323 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 862/1750] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:34.307364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a167080-0b55-4765-bd90-8f8e54e60b8a service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:34.319043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d92d00c0-547d-486b-9297-ce1b1086363d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36b87ad6b4fb48f09393d9c561371fd1'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:34.322700 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d92d00c0-547d-486b-9297-ce1b1086363d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36b87ad6b4fb48f09393d9c561371fd1'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:34.348515 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d92d00c0-547d-486b-9297-ce1b1086363d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36b87ad6b4fb48f09393d9c561371fd1'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:34.351744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d92d00c0-547d-486b-9297-ce1b1086363d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36b87ad6b4fb48f09393d9c561371fd1'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:34.366919 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a167080-0b55-4765-bd90-8f8e54e60b8a service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:34.393663 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7d74b-a7f4-488f-b679-5df888108f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:34.394771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7d74b-a7f4-488f-b679-5df888108f5a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:34.395276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7d74b-a7f4-488f-b679-5df888108f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:34.395795 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7d74b-a7f4-488f-b679-5df888108f5a admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:34.396878 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7d74b-a7f4-488f-b679-5df888108f5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:34.400847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7d74b-a7f4-488f-b679-5df888108f5a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:34.423077 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 889/1751] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:33 2022] DELETE /identity/v3/users/36b87ad6b4fb48f09393d9c561371fd1 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:34.424946 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d00c0-547d-486b-9297-ce1b1086363d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:34.429584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d92d00c0-547d-486b-9297-ce1b1086363d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:34.470084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 863/1752] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:34 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:34.471956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7d74b-a7f4-488f-b679-5df888108f5a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:34.478017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d9d7d74b-a7f4-488f-b679-5df888108f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:34.529095 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7d74b-a7f4-488f-b679-5df888108f5a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:34.620767 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:34.628849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d00c0-547d-486b-9297-ce1b1086363d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:34.646376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e9b20-07a2-4d52-89aa-eb91db4d690e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:34.647212 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e9b20-07a2-4d52-89aa-eb91db4d690e service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:34.647498 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e9b20-07a2-4d52-89aa-eb91db4d690e service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:34.648612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e9b20-07a2-4d52-89aa-eb91db4d690e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:34.653041 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b85e9b20-07a2-4d52-89aa-eb91db4d690e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:34.742356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e9b20-07a2-4d52-89aa-eb91db4d690e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:34.742356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e9b20-07a2-4d52-89aa-eb91db4d690e service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:34.766783 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 890/1753] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:34.826273 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2593ac-3637-46c5-acde-0c55cf07ffde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:34.826977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2593ac-3637-46c5-acde-0c55cf07ffde admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:34.827265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2593ac-3637-46c5-acde-0c55cf07ffde admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:34.827550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2593ac-3637-46c5-acde-0c55cf07ffde admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:34.828276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2593ac-3637-46c5-acde-0c55cf07ffde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:34.838831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2593ac-3637-46c5-acde-0c55cf07ffde admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:34.881264 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e9b20-07a2-4d52-89aa-eb91db4d690e service nova] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:34.895625 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 864/1754] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:34.905738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e9b20-07a2-4d52-89aa-eb91db4d690e service nova] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:34.916039 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158d3f4-2b97-48a5-b54b-c91825521431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:34.916747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158d3f4-2b97-48a5-b54b-c91825521431 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:34.917035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158d3f4-2b97-48a5-b54b-c91825521431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:34.917341 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158d3f4-2b97-48a5-b54b-c91825521431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:34.920301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158d3f4-2b97-48a5-b54b-c91825521431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:34.921999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158d3f4-2b97-48a5-b54b-c91825521431 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:34.958381 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 891/1755] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:34.961882 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2593ac-3637-46c5-acde-0c55cf07ffde admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:34.980473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2593ac-3637-46c5-acde-0c55cf07ffde admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:34.990651 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580ce84-265a-4b7c-b381-68beca40b8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:34.991698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580ce84-265a-4b7c-b381-68beca40b8e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:34.992095 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580ce84-265a-4b7c-b381-68beca40b8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:34.992479 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580ce84-265a-4b7c-b381-68beca40b8e8 admin admin] PATH_INFO: `/v3/projects/6f4961f1bbd8471c99eb298ac785eb5d/users/7798da08bc8646dd93d4582014c929ae/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:35.034448 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580ce84-265a-4b7c-b381-68beca40b8e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f4961f1bbd8471c99eb298ac785eb5d, user_id=7798da08bc8646dd93d4582014c929ae, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:35.037988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580ce84-265a-4b7c-b381-68beca40b8e8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:35.042835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:35.089632 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 865/1756] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:34 2022] PUT /identity/v3/projects/6f4961f1bbd8471c99eb298ac785eb5d/users/7798da08bc8646dd93d4582014c929ae/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:35.095681 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3c909-22de-4843-9a14-4d17fb185114 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:35.095990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3c909-22de-4843-9a14-4d17fb185114 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:35.096275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3c909-22de-4843-9a14-4d17fb185114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:35.151483 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-adf3c909-22de-4843-9a14-4d17fb185114 None None] MFA Rules not processed for user `7798da08bc8646dd93d4582014c929ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:35.152532 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-adf3c909-22de-4843-9a14-4d17fb185114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:35.443734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 892/1757] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:35.480667 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580ce84-265a-4b7c-b381-68beca40b8e8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:35.483203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1580ce84-265a-4b7c-b381-68beca40b8e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:35.793240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf3c909-22de-4843-9a14-4d17fb185114 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:35.796188 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-adf3c909-22de-4843-9a14-4d17fb185114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:35.798483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580ce84-265a-4b7c-b381-68beca40b8e8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:35.816085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d00b1-4f56-4384-b872-fe751e68c9c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '215c6f6f1aad486f85ce20590d889cbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:35.818600 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d00b1-4f56-4384-b872-fe751e68c9c2 service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:35.822072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d00b1-4f56-4384-b872-fe751e68c9c2 service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:35.824250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d00b1-4f56-4384-b872-fe751e68c9c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:35.862424 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d00b1-4f56-4384-b872-fe751e68c9c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:35.878592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d00b1-4f56-4384-b872-fe751e68c9c2 service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:35.971300 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 866/1758] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:36.155030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf3c909-22de-4843-9a14-4d17fb185114 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:36.174068 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e2b43-4fb3-4440-9ae9-a0e97a562fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:36.175257 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e2b43-4fb3-4440-9ae9-a0e97a562fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:36.175911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e2b43-4fb3-4440-9ae9-a0e97a562fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:36.175911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e2b43-4fb3-4440-9ae9-a0e97a562fb4 admin admin] PATH_INFO: `/v3/projects/62a22e9e1669461b9ec9e03a523bf7f8` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:36.178757 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e2b43-4fb3-4440-9ae9-a0e97a562fb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a22e9e1669461b9ec9e03a523bf7f8)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:36.182509 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e2b43-4fb3-4440-9ae9-a0e97a562fb4 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:36.235676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:36.314528 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8d5e2b43-4fb3-4440-9ae9-a0e97a562fb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a22e9e1669461b9ec9e03a523bf7f8', 'request_id': 'req-8d5e2b43-4fb3-4440-9ae9-a0e97a562fb4'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:36.330112 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8d5e2b43-4fb3-4440-9ae9-a0e97a562fb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a22e9e1669461b9ec9e03a523bf7f8', 'request_id': 'req-8d5e2b43-4fb3-4440-9ae9-a0e97a562fb4'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:36.354147 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8d5e2b43-4fb3-4440-9ae9-a0e97a562fb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a22e9e1669461b9ec9e03a523bf7f8', 'request_id': 'req-8d5e2b43-4fb3-4440-9ae9-a0e97a562fb4'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:36.383744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8d5e2b43-4fb3-4440-9ae9-a0e97a562fb4 admin admin] The token cache is being invalidate because project 62a22e9e1669461b9ec9e03a523bf7f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:36.385518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8d5e2b43-4fb3-4440-9ae9-a0e97a562fb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:36.386435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8d5e2b43-4fb3-4440-9ae9-a0e97a562fb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:36.403583 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 893/1759] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:35 2022] DELETE /identity/v3/projects/62a22e9e1669461b9ec9e03a523bf7f8 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:36.406835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00fa48-75be-47f8-94fd-47b03ad9de66 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:36.409421 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00fa48-75be-47f8-94fd-47b03ad9de66 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:36.409984 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00fa48-75be-47f8-94fd-47b03ad9de66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:36.487828 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4b00fa48-75be-47f8-94fd-47b03ad9de66 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:36.496696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4b00fa48-75be-47f8-94fd-47b03ad9de66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:36.721583 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6dd97-402a-460b-9b72-aa9815e13af4 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:36.721876 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6dd97-402a-460b-9b72-aa9815e13af4 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:36.723092 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6dd97-402a-460b-9b72-aa9815e13af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:36.732193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 867/1760] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:36.834189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-c5a6dd97-402a-460b-9b72-aa9815e13af4 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:36.846170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c5a6dd97-402a-460b-9b72-aa9815e13af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:36.879988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00fa48-75be-47f8-94fd-47b03ad9de66 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:36.885983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 894/1761] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:36.900817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00fa48-75be-47f8-94fd-47b03ad9de66 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:36.916276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf976e9-64cb-456e-b61a-22477dcacad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:36.918533 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf976e9-64cb-456e-b61a-22477dcacad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:36.918919 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf976e9-64cb-456e-b61a-22477dcacad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:36.920489 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf976e9-64cb-456e-b61a-22477dcacad0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:36.921324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf976e9-64cb-456e-b61a-22477dcacad0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:36.928970 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf976e9-64cb-456e-b61a-22477dcacad0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:36.937970 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf976e9-64cb-456e-b61a-22477dcacad0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:36.959197 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 868/1762] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:30:36.964871 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6dd97-402a-460b-9b72-aa9815e13af4 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:36.983734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6dd97-402a-460b-9b72-aa9815e13af4 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:36.989563 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ee853-4989-43b4-b306-0083dc3c7ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:36.990252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ee853-4989-43b4-b306-0083dc3c7ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:36.990536 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ee853-4989-43b4-b306-0083dc3c7ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:36.990816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ee853-4989-43b4-b306-0083dc3c7ef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:36.991535 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ee853-4989-43b4-b306-0083dc3c7ef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:36.993555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ee853-4989-43b4-b306-0083dc3c7ef3 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:37.115705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 895/1763] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:36 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:37.121267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf976e9-64cb-456e-b61a-22477dcacad0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:37.144477 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf976e9-64cb-456e-b61a-22477dcacad0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:37.499809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33faa0-4004-44cd-a459-d210bb97c88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:37.503525 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33faa0-4004-44cd-a459-d210bb97c88e admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:37.504045 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33faa0-4004-44cd-a459-d210bb97c88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:37.504898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33faa0-4004-44cd-a459-d210bb97c88e admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:37.508160 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33faa0-4004-44cd-a459-d210bb97c88e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:37.514892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33faa0-4004-44cd-a459-d210bb97c88e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:37.547817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:37.569517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 869/1764] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:37 2022] POST /identity/v3/users => generated 286 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:37.572989 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ee853-4989-43b4-b306-0083dc3c7ef3 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:37.589499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ee853-4989-43b4-b306-0083dc3c7ef3 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:37.633641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f86d0-5ca8-47b9-960b-5133c590b085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:37.646592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f86d0-5ca8-47b9-960b-5133c590b085 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:37.647673 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f86d0-5ca8-47b9-960b-5133c590b085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:37.660450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f86d0-5ca8-47b9-960b-5133c590b085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:37.660450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f86d0-5ca8-47b9-960b-5133c590b085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:37.667210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f86d0-5ca8-47b9-960b-5133c590b085 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:37.760444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33faa0-4004-44cd-a459-d210bb97c88e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:37.771620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33faa0-4004-44cd-a459-d210bb97c88e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:37.788298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 896/1765] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:37.795450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710a836-fa06-4630-8d39-7e6d2033f772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:37.798831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710a836-fa06-4630-8d39-7e6d2033f772 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:37.800127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710a836-fa06-4630-8d39-7e6d2033f772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:37.801276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710a836-fa06-4630-8d39-7e6d2033f772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:37.801276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710a836-fa06-4630-8d39-7e6d2033f772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:37.812088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710a836-fa06-4630-8d39-7e6d2033f772 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:37.882314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f86d0-5ca8-47b9-960b-5133c590b085 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:37.886010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 870/1766] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:30:37.908329 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f86d0-5ca8-47b9-960b-5133c590b085 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:37.932495 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4974b-6b9c-4193-9bd1-714a030b6d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:37.936800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4974b-6b9c-4193-9bd1-714a030b6d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:37.943025 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4974b-6b9c-4193-9bd1-714a030b6d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:37.948258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4974b-6b9c-4193-9bd1-714a030b6d97 admin admin] PATH_INFO: `/v3/projects/b01436b67f11490698d23fe930a4a3a0/users/cd5e6483f72c4b068630a894a28ce421/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:38.064905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4974b-6b9c-4193-9bd1-714a030b6d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01436b67f11490698d23fe930a4a3a0, user_id=cd5e6483f72c4b068630a894a28ce421, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:38.078630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4974b-6b9c-4193-9bd1-714a030b6d97 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:38.143022 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 897/1767] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:37 2022] PUT /identity/v3/projects/b01436b67f11490698d23fe930a4a3a0/users/cd5e6483f72c4b068630a894a28ce421/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:38.163060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae825c-1f5f-49d6-a9bc-08daa28b7f54 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:38.163691 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae825c-1f5f-49d6-a9bc-08daa28b7f54 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:38.164148 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae825c-1f5f-49d6-a9bc-08daa28b7f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:38.273250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8dae825c-1f5f-49d6-a9bc-08daa28b7f54 None None] MFA Rules not processed for user `cd5e6483f72c4b068630a894a28ce421`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:38.273780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8dae825c-1f5f-49d6-a9bc-08daa28b7f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:38.358491 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190be591-fd59-48c1-a80c-bf3a5caad071 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:38.358989 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190be591-fd59-48c1-a80c-bf3a5caad071 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:38.362060 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190be591-fd59-48c1-a80c-bf3a5caad071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:38.638098 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 871/1768] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:38.642263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae825c-1f5f-49d6-a9bc-08daa28b7f54 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:38.690182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-190be591-fd59-48c1-a80c-bf3a5caad071 None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:38.699475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-190be591-fd59-48c1-a80c-bf3a5caad071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:38.924596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae825c-1f5f-49d6-a9bc-08daa28b7f54 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:38.943799 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d51d18-98a2-470b-a9a0-4e98d275eb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:38.945179 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d51d18-98a2-470b-a9a0-4e98d275eb7e admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:38.952686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d51d18-98a2-470b-a9a0-4e98d275eb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:38.957797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d51d18-98a2-470b-a9a0-4e98d275eb7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:38.960985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d51d18-98a2-470b-a9a0-4e98d275eb7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:38.972620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 898/1769] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:30:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 616 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:38.974441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d51d18-98a2-470b-a9a0-4e98d275eb7e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:38.980821 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acadd3e-7517-4f92-89b0-6c02f828e14f None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:38.982005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acadd3e-7517-4f92-89b0-6c02f828e14f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:38.983068 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acadd3e-7517-4f92-89b0-6c02f828e14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:39.073948 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-2acadd3e-7517-4f92-89b0-6c02f828e14f None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:39.081221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2acadd3e-7517-4f92-89b0-6c02f828e14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:39.091097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 872/1770] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:38 2022] POST /identity/v3/projects => generated 371 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:30:39.095377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d51d18-98a2-470b-a9a0-4e98d275eb7e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:39.163677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:39.199773 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d51d18-98a2-470b-a9a0-4e98d275eb7e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:39.205277 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e790017-c0b4-459b-b096-73116a2efecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:39.206276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e790017-c0b4-459b-b096-73116a2efecb admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:39.206657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e790017-c0b4-459b-b096-73116a2efecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:39.207259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e790017-c0b4-459b-b096-73116a2efecb admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:39.208072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e790017-c0b4-459b-b096-73116a2efecb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:39.213647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e790017-c0b4-459b-b096-73116a2efecb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:39.222541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:39.281473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 873/1771] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:39 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:39.283594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e790017-c0b4-459b-b096-73116a2efecb admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:39.292494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0e790017-c0b4-459b-b096-73116a2efecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:39.327842 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 899/1772] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:30:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:39.329703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acadd3e-7517-4f92-89b0-6c02f828e14f None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:39.362455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acadd3e-7517-4f92-89b0-6c02f828e14f None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:39.377469 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7409baa-37d2-45fa-b9e1-0fffd3623c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:39.378169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7409baa-37d2-45fa-b9e1-0fffd3623c60 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:39.380545 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7409baa-37d2-45fa-b9e1-0fffd3623c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:39.380887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7409baa-37d2-45fa-b9e1-0fffd3623c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:39.381683 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7409baa-37d2-45fa-b9e1-0fffd3623c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:39.387205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7409baa-37d2-45fa-b9e1-0fffd3623c60 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:39.413839 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 900/1773] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:39.418347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7409baa-37d2-45fa-b9e1-0fffd3623c60 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:39.421029 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e7409baa-37d2-45fa-b9e1-0fffd3623c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:39.528029 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e790017-c0b4-459b-b096-73116a2efecb admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:39.543999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f687c-66be-4abf-9218-0db49399a376 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:39.549564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f687c-66be-4abf-9218-0db49399a376 service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:39.549564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f687c-66be-4abf-9218-0db49399a376 service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:39.549564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f687c-66be-4abf-9218-0db49399a376 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:39.585958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f687c-66be-4abf-9218-0db49399a376 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:39.588283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f687c-66be-4abf-9218-0db49399a376 service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:39.609127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 874/1774] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:30:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:30:39.613121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f687c-66be-4abf-9218-0db49399a376 service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:39.657128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7409baa-37d2-45fa-b9e1-0fffd3623c60 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:39.667833 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f687c-66be-4abf-9218-0db49399a376 service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:39.674117 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc79e88-bde7-413a-8f6c-5bbcd74b6ac5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:39.674846 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc79e88-bde7-413a-8f6c-5bbcd74b6ac5 service swift] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:39.675134 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc79e88-bde7-413a-8f6c-5bbcd74b6ac5 service swift] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:39.675411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc79e88-bde7-413a-8f6c-5bbcd74b6ac5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:39.699978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13345dfc-e926-49a2-9524-60aa0ec3fe4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:39.701478 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13345dfc-e926-49a2-9524-60aa0ec3fe4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:39.706924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13345dfc-e926-49a2-9524-60aa0ec3fe4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:39.707445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13345dfc-e926-49a2-9524-60aa0ec3fe4c admin admin] PATH_INFO: `/v3/projects/b8c7289737cd4b3783ef6aa14394622b/users/62858da8658144eb92b6691cee0b3eda/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:39.715242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc79e88-bde7-413a-8f6c-5bbcd74b6ac5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:39.719953 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc79e88-bde7-413a-8f6c-5bbcd74b6ac5 service swift] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:39.769526 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 901/1775] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:30:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:30:39.852248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13345dfc-e926-49a2-9524-60aa0ec3fe4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8c7289737cd4b3783ef6aa14394622b, user_id=62858da8658144eb92b6691cee0b3eda, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:39.874801 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13345dfc-e926-49a2-9524-60aa0ec3fe4c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:39.960616 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 875/1776] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:39 2022] PUT /identity/v3/projects/b8c7289737cd4b3783ef6aa14394622b/users/62858da8658144eb92b6691cee0b3eda/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:39.967735 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc79e88-bde7-413a-8f6c-5bbcd74b6ac5 service swift] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:40.074594 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc79e88-bde7-413a-8f6c-5bbcd74b6ac5 service swift] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:40.395777 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2c50f-884f-49d1-b330-20190f239806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:40.407952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2c50f-884f-49d1-b330-20190f239806 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:40.407952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2c50f-884f-49d1-b330-20190f239806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:40.407952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2c50f-884f-49d1-b330-20190f239806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:40.407952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2c50f-884f-49d1-b330-20190f239806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:40.434316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2c50f-884f-49d1-b330-20190f239806 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:40.516715 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 902/1777] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 552 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:40.530389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13345dfc-e926-49a2-9524-60aa0ec3fe4c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:40.548949 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13345dfc-e926-49a2-9524-60aa0ec3fe4c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:40.583095 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac97431-ef74-4217-9705-be897d994c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:40.591225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac97431-ef74-4217-9705-be897d994c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:40.600366 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac97431-ef74-4217-9705-be897d994c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:40.607528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac97431-ef74-4217-9705-be897d994c69 admin admin] PATH_INFO: `/v3/projects/b8c7289737cd4b3783ef6aa14394622b/users/62858da8658144eb92b6691cee0b3eda/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:40.610293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac97431-ef74-4217-9705-be897d994c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8c7289737cd4b3783ef6aa14394622b, user_id=62858da8658144eb92b6691cee0b3eda, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:40.613378 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac97431-ef74-4217-9705-be897d994c69 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:40.630224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2c50f-884f-49d1-b330-20190f239806 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:40.632328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-79b2c50f-884f-49d1-b330-20190f239806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:40.674773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2c50f-884f-49d1-b330-20190f239806 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:40.694647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 876/1778] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:40 2022] PUT /identity/v3/projects/b8c7289737cd4b3783ef6aa14394622b/users/62858da8658144eb92b6691cee0b3eda/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:40.700811 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378968fe-acdd-4be5-8277-e23bef5c3b22 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:40.701444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378968fe-acdd-4be5-8277-e23bef5c3b22 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:40.702016 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378968fe-acdd-4be5-8277-e23bef5c3b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:40.796205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-378968fe-acdd-4be5-8277-e23bef5c3b22 None None] MFA Rules not processed for user `62858da8658144eb92b6691cee0b3eda`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:40.810615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-378968fe-acdd-4be5-8277-e23bef5c3b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:40.875410 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:40.964408 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0429e-cba3-44d0-a08a-0b89cbca1869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:40.968248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0429e-cba3-44d0-a08a-0b89cbca1869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:40.968585 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0429e-cba3-44d0-a08a-0b89cbca1869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:40.968915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0429e-cba3-44d0-a08a-0b89cbca1869 admin admin] PATH_INFO: `/v3/users/9178a544490b4d879cd338104c9b46c0` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:40.979683 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0429e-cba3-44d0-a08a-0b89cbca1869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9178a544490b4d879cd338104c9b46c0)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:40.986040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0429e-cba3-44d0-a08a-0b89cbca1869 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:41.169181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 877/1779] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:40 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:41.174494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378968fe-acdd-4be5-8277-e23bef5c3b22 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:41.185502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-378968fe-acdd-4be5-8277-e23bef5c3b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:41.214934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378968fe-acdd-4be5-8277-e23bef5c3b22 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:41.227577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-f3c0429e-cba3-44d0-a08a-0b89cbca1869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9178a544490b4d879cd338104c9b46c0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:41.234501 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bb039-0c7a-450e-9afb-5bb7bfcc4985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:41.235433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bb039-0c7a-450e-9afb-5bb7bfcc4985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:41.235674 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bb039-0c7a-450e-9afb-5bb7bfcc4985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:41.235952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bb039-0c7a-450e-9afb-5bb7bfcc4985 admin admin] PATH_INFO: `/v3/users/8d8ed9319c1740eb92b05a1453d5599e` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:41.259370 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bb039-0c7a-450e-9afb-5bb7bfcc4985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d8ed9319c1740eb92b05a1453d5599e)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:41.264131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bb039-0c7a-450e-9afb-5bb7bfcc4985 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:41.267301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:41.370367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-f3c0429e-cba3-44d0-a08a-0b89cbca1869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9178a544490b4d879cd338104c9b46c0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:41.371377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-f3c0429e-cba3-44d0-a08a-0b89cbca1869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9178a544490b4d879cd338104c9b46c0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:41.412827 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-f3c0429e-cba3-44d0-a08a-0b89cbca1869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9178a544490b4d879cd338104c9b46c0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:41.420464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-f3c0429e-cba3-44d0-a08a-0b89cbca1869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9178a544490b4d879cd338104c9b46c0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:41.488975 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 903/1780] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:40 2022] DELETE /identity/v3/users/9178a544490b4d879cd338104c9b46c0 => generated 0 bytes in 859 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:41.492909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0429e-cba3-44d0-a08a-0b89cbca1869 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:41.499342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f3c0429e-cba3-44d0-a08a-0b89cbca1869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:41.564344 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-750bb039-0c7a-450e-9afb-5bb7bfcc4985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d8ed9319c1740eb92b05a1453d5599e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:41.668785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-750bb039-0c7a-450e-9afb-5bb7bfcc4985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d8ed9319c1740eb92b05a1453d5599e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:41.679623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-750bb039-0c7a-450e-9afb-5bb7bfcc4985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d8ed9319c1740eb92b05a1453d5599e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:41.688183 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-750bb039-0c7a-450e-9afb-5bb7bfcc4985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d8ed9319c1740eb92b05a1453d5599e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:41.723469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-750bb039-0c7a-450e-9afb-5bb7bfcc4985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d8ed9319c1740eb92b05a1453d5599e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:41.764994 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 878/1781] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:41 2022] DELETE /identity/v3/users/8d8ed9319c1740eb92b05a1453d5599e => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:41.780750 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2020f82-2dba-4fd7-aaed-1cb4e92f0b7a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:41.781066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2020f82-2dba-4fd7-aaed-1cb4e92f0b7a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:41.781419 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2020f82-2dba-4fd7-aaed-1cb4e92f0b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:41.876422 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d2020f82-2dba-4fd7-aaed-1cb4e92f0b7a None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:41.891550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d2020f82-2dba-4fd7-aaed-1cb4e92f0b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:41.907235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0429e-cba3-44d0-a08a-0b89cbca1869 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:42.218386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60839104-4fcb-4a36-a17e-7cf20cc54fb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:42.219039 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60839104-4fcb-4a36-a17e-7cf20cc54fb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:42.219325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60839104-4fcb-4a36-a17e-7cf20cc54fb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:42.219618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60839104-4fcb-4a36-a17e-7cf20cc54fb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:42.224104 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-60839104-4fcb-4a36-a17e-7cf20cc54fb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:42.343103 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 879/1782] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 577 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:42.344776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3122f-c0d3-4bb2-beef-6c36d8abe5db None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:42.345067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3122f-c0d3-4bb2-beef-6c36d8abe5db None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:42.345467 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3122f-c0d3-4bb2-beef-6c36d8abe5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:42.352058 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:42.438847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-70f3122f-c0d3-4bb2-beef-6c36d8abe5db None None] MFA Rules not processed for user `ddb5e1dec1904ae183e131ba95ed1ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:42.443927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-70f3122f-c0d3-4bb2-beef-6c36d8abe5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:42.591637 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60839104-4fcb-4a36-a17e-7cf20cc54fb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:42.596379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60839104-4fcb-4a36-a17e-7cf20cc54fb3 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:42.662336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 904/1783] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:41 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 1172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:42.662862 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477959ca-e39f-47f3-8916-e22eb9912f65 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:42.663198 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477959ca-e39f-47f3-8916-e22eb9912f65 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:42.663910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477959ca-e39f-47f3-8916-e22eb9912f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:42.783260 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-477959ca-e39f-47f3-8916-e22eb9912f65 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:42.785764 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-477959ca-e39f-47f3-8916-e22eb9912f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:42.829060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 880/1784] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:42.830215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3122f-c0d3-4bb2-beef-6c36d8abe5db None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:42.836169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 905/1785] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:42.839840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d85c50-4690-481e-8ce2-7b87c9025114 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:42.842269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d85c50-4690-481e-8ce2-7b87c9025114 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:42.842269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d85c50-4690-481e-8ce2-7b87c9025114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:42.848519 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-70f3122f-c0d3-4bb2-beef-6c36d8abe5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:42.908149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-58d85c50-4690-481e-8ce2-7b87c9025114 None None] MFA Rules not processed for user `51eb5efc37ca4942831bc4d046f54da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:42.919192 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-58d85c50-4690-481e-8ce2-7b87c9025114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:43.027217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:43.231894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 906/1786] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:43.233895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d85c50-4690-481e-8ce2-7b87c9025114 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:43.235491 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-58d85c50-4690-481e-8ce2-7b87c9025114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:43.278451 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3122f-c0d3-4bb2-beef-6c36d8abe5db None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:43.293980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d95145-7c7d-40a0-8987-e86016d79d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:43.294240 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d95145-7c7d-40a0-8987-e86016d79d52 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:43.294525 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d95145-7c7d-40a0-8987-e86016d79d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:43.294797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d95145-7c7d-40a0-8987-e86016d79d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:43.309166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d85c50-4690-481e-8ce2-7b87c9025114 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:43.321986 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa79a68-b512-4f45-8612-a41f30b2deb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:43.325049 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa79a68-b512-4f45-8612-a41f30b2deb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:43.325367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa79a68-b512-4f45-8612-a41f30b2deb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:43.325650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa79a68-b512-4f45-8612-a41f30b2deb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:43.335273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-afa79a68-b512-4f45-8612-a41f30b2deb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:43.409121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d95145-7c7d-40a0-8987-e86016d79d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:43.413684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d95145-7c7d-40a0-8987-e86016d79d52 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:43.454461 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 881/1787] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:43.455493 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d95145-7c7d-40a0-8987-e86016d79d52 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:43.492091 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d95145-7c7d-40a0-8987-e86016d79d52 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:43.506934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465722f2-93c0-40f3-a326-a0de13567530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:43.506934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465722f2-93c0-40f3-a326-a0de13567530 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:43.507187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465722f2-93c0-40f3-a326-a0de13567530 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:43.507427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465722f2-93c0-40f3-a326-a0de13567530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:43.549193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465722f2-93c0-40f3-a326-a0de13567530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:43.551478 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465722f2-93c0-40f3-a326-a0de13567530 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:43.613046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 882/1788] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:43.623866 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0b027-f6fd-4dd2-857b-a4d36f0f9896 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:43.625182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0b027-f6fd-4dd2-857b-a4d36f0f9896 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:43.625182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0b027-f6fd-4dd2-857b-a4d36f0f9896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:43.629028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa79a68-b512-4f45-8612-a41f30b2deb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:43.630551 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa79a68-b512-4f45-8612-a41f30b2deb4 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:43.679025 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 907/1789] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:43.767970 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0ba0b027-f6fd-4dd2-857b-a4d36f0f9896 None None] MFA Rules not processed for user `5507329a6bb84cfbbdac99b85f346ab8`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:43.776424 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0ba0b027-f6fd-4dd2-857b-a4d36f0f9896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:43.807365 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023021e6-8193-46de-b836-74fcd78fc95f None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:43.809167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023021e6-8193-46de-b836-74fcd78fc95f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:43.810316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023021e6-8193-46de-b836-74fcd78fc95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:43.895424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-023021e6-8193-46de-b836-74fcd78fc95f None None] MFA Rules not processed for user `369bdaded0534142b03ad3cf43a70881`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:43.898360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-023021e6-8193-46de-b836-74fcd78fc95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:43.944612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 908/1790] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Jul 19 15:30:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:44.047858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 883/1791] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:44.226885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023021e6-8193-46de-b836-74fcd78fc95f None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:44.252076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023021e6-8193-46de-b836-74fcd78fc95f None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:44.285665 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738b292-57e8-416a-a023-3c53574f5d92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:44.286345 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738b292-57e8-416a-a023-3c53574f5d92 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:44.286648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738b292-57e8-416a-a023-3c53574f5d92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:44.287184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738b292-57e8-416a-a023-3c53574f5d92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:44.304110 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738b292-57e8-416a-a023-3c53574f5d92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:44.308163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738b292-57e8-416a-a023-3c53574f5d92 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:44.328145 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 909/1792] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:30:45.074256 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0b027-f6fd-4dd2-857b-a4d36f0f9896 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:45.076263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0ba0b027-f6fd-4dd2-857b-a4d36f0f9896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:45.120216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0b027-f6fd-4dd2-857b-a4d36f0f9896 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:45.135006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458282b5-2c89-42a2-a9df-c869b57134e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:45.136118 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458282b5-2c89-42a2-a9df-c869b57134e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:45.136429 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458282b5-2c89-42a2-a9df-c869b57134e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:45.136745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458282b5-2c89-42a2-a9df-c869b57134e3 admin admin] PATH_INFO: `/v3/projects/02f097a87fa6490a886516d83fe95c55` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:45.138256 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458282b5-2c89-42a2-a9df-c869b57134e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02f097a87fa6490a886516d83fe95c55)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:45.142177 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458282b5-2c89-42a2-a9df-c869b57134e3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:45.359285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-458282b5-2c89-42a2-a9df-c869b57134e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02f097a87fa6490a886516d83fe95c55', 'request_id': 'req-458282b5-2c89-42a2-a9df-c869b57134e3'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:45.375518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738b292-57e8-416a-a023-3c53574f5d92 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:45.379005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9738b292-57e8-416a-a023-3c53574f5d92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:45.400534 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-458282b5-2c89-42a2-a9df-c869b57134e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02f097a87fa6490a886516d83fe95c55', 'request_id': 'req-458282b5-2c89-42a2-a9df-c869b57134e3'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:45.429654 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738b292-57e8-416a-a023-3c53574f5d92 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:45.455055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-458282b5-2c89-42a2-a9df-c869b57134e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02f097a87fa6490a886516d83fe95c55', 'request_id': 'req-458282b5-2c89-42a2-a9df-c869b57134e3'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:45.511724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-458282b5-2c89-42a2-a9df-c869b57134e3 admin admin] The token cache is being invalidate because project 02f097a87fa6490a886516d83fe95c55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:45.512040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-458282b5-2c89-42a2-a9df-c869b57134e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:45.514171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-458282b5-2c89-42a2-a9df-c869b57134e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:45.526961 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 884/1793] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:45 2022] DELETE /identity/v3/projects/02f097a87fa6490a886516d83fe95c55 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:45.531364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458282b5-2c89-42a2-a9df-c869b57134e3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:45.538858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-458282b5-2c89-42a2-a9df-c869b57134e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:45.669976 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:45.753263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15a638-f616-45ca-910c-15f2e772c433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:45.754449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15a638-f616-45ca-910c-15f2e772c433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:45.766243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15a638-f616-45ca-910c-15f2e772c433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:45.767252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15a638-f616-45ca-910c-15f2e772c433 admin admin] PATH_INFO: `/v3/projects/73e5187bf3b74695a127aa9a0aa5b12d` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:45.771948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:45.775283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15a638-f616-45ca-910c-15f2e772c433 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73e5187bf3b74695a127aa9a0aa5b12d)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:45.784654 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15a638-f616-45ca-910c-15f2e772c433 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:45.886238 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458282b5-2c89-42a2-a9df-c869b57134e3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:45.929722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da18b4-d0db-4ff6-ab89-cc56ba147fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:45.936022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da18b4-d0db-4ff6-ab89-cc56ba147fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:45.936673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da18b4-d0db-4ff6-ab89-cc56ba147fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:45.937409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da18b4-d0db-4ff6-ab89-cc56ba147fc8 admin admin] PATH_INFO: `/v3/users/a7c6ee18f08a41de8489b09dccfc73f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:45.940695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da18b4-d0db-4ff6-ab89-cc56ba147fc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7c6ee18f08a41de8489b09dccfc73f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:45.974990 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da18b4-d0db-4ff6-ab89-cc56ba147fc8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:46.102232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-3c15a638-f616-45ca-910c-15f2e772c433 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73e5187bf3b74695a127aa9a0aa5b12d', 'request_id': 'req-3c15a638-f616-45ca-910c-15f2e772c433'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:46.133096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-3c15a638-f616-45ca-910c-15f2e772c433 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73e5187bf3b74695a127aa9a0aa5b12d', 'request_id': 'req-3c15a638-f616-45ca-910c-15f2e772c433'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:46.202735 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-3c15a638-f616-45ca-910c-15f2e772c433 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73e5187bf3b74695a127aa9a0aa5b12d', 'request_id': 'req-3c15a638-f616-45ca-910c-15f2e772c433'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:46.309536 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-3c15a638-f616-45ca-910c-15f2e772c433 admin admin] The token cache is being invalidate because project 73e5187bf3b74695a127aa9a0aa5b12d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:46.311328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-3c15a638-f616-45ca-910c-15f2e772c433 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:46.313419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-3c15a638-f616-45ca-910c-15f2e772c433 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:46.323910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 910/1794] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:45 2022] DELETE /identity/v3/projects/73e5187bf3b74695a127aa9a0aa5b12d => generated 0 bytes in 945 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:46.326530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15a638-f616-45ca-910c-15f2e772c433 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:46.338672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3c15a638-f616-45ca-910c-15f2e772c433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:46.339371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-56da18b4-d0db-4ff6-ab89-cc56ba147fc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7c6ee18f08a41de8489b09dccfc73f6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:46.439768 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-56da18b4-d0db-4ff6-ab89-cc56ba147fc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7c6ee18f08a41de8489b09dccfc73f6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:46.443383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-56da18b4-d0db-4ff6-ab89-cc56ba147fc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7c6ee18f08a41de8489b09dccfc73f6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:46.451814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-56da18b4-d0db-4ff6-ab89-cc56ba147fc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7c6ee18f08a41de8489b09dccfc73f6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:46.499507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-56da18b4-d0db-4ff6-ab89-cc56ba147fc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7c6ee18f08a41de8489b09dccfc73f6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:46.574963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 885/1795] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:45 2022] DELETE /identity/v3/users/a7c6ee18f08a41de8489b09dccfc73f6 => generated 0 bytes in 1044 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:46.581123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15a638-f616-45ca-910c-15f2e772c433 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:46.850322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd2741-9218-44ed-b278-44422ef6cdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:46.851744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd2741-9218-44ed-b278-44422ef6cdaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:46.851744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd2741-9218-44ed-b278-44422ef6cdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:46.851963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd2741-9218-44ed-b278-44422ef6cdaf admin admin] PATH_INFO: `/v3/users/05c1e44e62484930b9cef41213d0d20b` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:46.858389 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd2741-9218-44ed-b278-44422ef6cdaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=05c1e44e62484930b9cef41213d0d20b)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:46.860506 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd2741-9218-44ed-b278-44422ef6cdaf admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:46.909887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:46.998305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-f0cd2741-9218-44ed-b278-44422ef6cdaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05c1e44e62484930b9cef41213d0d20b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:47.035233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-f0cd2741-9218-44ed-b278-44422ef6cdaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05c1e44e62484930b9cef41213d0d20b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:47.037227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-f0cd2741-9218-44ed-b278-44422ef6cdaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05c1e44e62484930b9cef41213d0d20b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:47.077194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-f0cd2741-9218-44ed-b278-44422ef6cdaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05c1e44e62484930b9cef41213d0d20b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:47.078975 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-f0cd2741-9218-44ed-b278-44422ef6cdaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05c1e44e62484930b9cef41213d0d20b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:47.117532 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 911/1796] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:46 2022] DELETE /identity/v3/users/05c1e44e62484930b9cef41213d0d20b => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:47.329605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e211770-17be-4890-a869-71825cca8127 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:47.329897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e211770-17be-4890-a869-71825cca8127 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:47.330173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e211770-17be-4890-a869-71825cca8127 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:47.331596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 886/1797] 10.222.0.42 () {54 vars in 921 bytes} [Tue Jul 19 15:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:30:47.579744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba01e08-c247-4918-9e26-5277ada0b66e None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:47.582192 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba01e08-c247-4918-9e26-5277ada0b66e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:47.582816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba01e08-c247-4918-9e26-5277ada0b66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:47.692357 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-fba01e08-c247-4918-9e26-5277ada0b66e None None] MFA Rules not processed for user `bff6a2d4bc734904b2615608a86b097d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:47.698938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-fba01e08-c247-4918-9e26-5277ada0b66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:47.936243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 912/1798] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Jul 19 15:30:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:48.021207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e211770-17be-4890-a869-71825cca8127 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:48.023572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3e211770-17be-4890-a869-71825cca8127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:48.431160 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e211770-17be-4890-a869-71825cca8127 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:48.452032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:48.455355 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:48.456588 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:48.456905 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c admin admin] PATH_INFO: `/v3/projects/db1189b9007e4b42976625151c757d3d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:48.480011 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c admin admin] RBAC: Authorizing `identity:delete_project(project_id=db1189b9007e4b42976625151c757d3d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:48.486360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:48.678602 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db1189b9007e4b42976625151c757d3d', 'request_id': 'req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:48.712599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db1189b9007e4b42976625151c757d3d', 'request_id': 'req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:48.778756 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:48.788097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db1189b9007e4b42976625151c757d3d', 'request_id': 'req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:48.811283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c admin admin] The token cache is being invalidate because project db1189b9007e4b42976625151c757d3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:48.811583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:48.812554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:48.815080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 887/1799] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:48 2022] DELETE /identity/v3/projects/db1189b9007e4b42976625151c757d3d => generated 0 bytes in 794 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:48.819421 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb88fd-b152-4a74-8be3-9522de7d2f11 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:48.820359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb88fd-b152-4a74-8be3-9522de7d2f11 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:48.820797 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb88fd-b152-4a74-8be3-9522de7d2f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:48.844617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ebdb88fd-b152-4a74-8be3-9522de7d2f11 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:48.845984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ebdb88fd-b152-4a74-8be3-9522de7d2f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:48.976825 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:48.979049 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:49.034991 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 913/1800] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:30:49.044832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b193a15-f817-4a00-955f-514bcec8b1e3 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.045336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b193a15-f817-4a00-955f-514bcec8b1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.045727 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b193a15-f817-4a00-955f-514bcec8b1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.066621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-6b193a15-f817-4a00-955f-514bcec8b1e3 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:49.067930 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-6b193a15-f817-4a00-955f-514bcec8b1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:49.077768 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 914/1801] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:30:49.081799 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b193a15-f817-4a00-955f-514bcec8b1e3 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:49.083130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f641f3a-90fe-43ad-90c7-43dd92f0d97c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:49.090918 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7106e-aeac-4954-a862-3b500adb7994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:49.091874 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b193a15-f817-4a00-955f-514bcec8b1e3 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:49.092549 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7106e-aeac-4954-a862-3b500adb7994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.092891 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7106e-aeac-4954-a862-3b500adb7994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.093109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7106e-aeac-4954-a862-3b500adb7994 admin admin] PATH_INFO: `/v3/projects/980b2412c0e34520bb0db01eb8f172e4` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.094393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7106e-aeac-4954-a862-3b500adb7994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=980b2412c0e34520bb0db01eb8f172e4)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:49.096894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7106e-aeac-4954-a862-3b500adb7994 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:49.097281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a873ba-4fa6-4c34-889a-d4559b85a555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:49.097956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a873ba-4fa6-4c34-889a-d4559b85a555 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.098238 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a873ba-4fa6-4c34-889a-d4559b85a555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.098514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a873ba-4fa6-4c34-889a-d4559b85a555 admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.099241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a873ba-4fa6-4c34-889a-d4559b85a555 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:49.099817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a873ba-4fa6-4c34-889a-d4559b85a555 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:49.101249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a873ba-4fa6-4c34-889a-d4559b85a555 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:49.116227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 915/1802] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:30:49.120175 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a873ba-4fa6-4c34-889a-d4559b85a555 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:49.132659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a873ba-4fa6-4c34-889a-d4559b85a555 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:49.139365 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a4189-e4bf-4360-b9b7-b5a1d7e5af80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:49.140036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a4189-e4bf-4360-b9b7-b5a1d7e5af80 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.140343 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a4189-e4bf-4360-b9b7-b5a1d7e5af80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.140581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a4189-e4bf-4360-b9b7-b5a1d7e5af80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.141327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a4189-e4bf-4360-b9b7-b5a1d7e5af80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:49.143253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a4189-e4bf-4360-b9b7-b5a1d7e5af80 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:49.155144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:49.176438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4dc7106e-aeac-4954-a862-3b500adb7994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '980b2412c0e34520bb0db01eb8f172e4', 'request_id': 'req-4dc7106e-aeac-4954-a862-3b500adb7994'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:49.193795 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 916/1803] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:49 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:30:49.194326 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4dc7106e-aeac-4954-a862-3b500adb7994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '980b2412c0e34520bb0db01eb8f172e4', 'request_id': 'req-4dc7106e-aeac-4954-a862-3b500adb7994'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:49.199315 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a4189-e4bf-4360-b9b7-b5a1d7e5af80 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:49.202425 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4dc7106e-aeac-4954-a862-3b500adb7994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '980b2412c0e34520bb0db01eb8f172e4', 'request_id': 'req-4dc7106e-aeac-4954-a862-3b500adb7994'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:49.223024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4dc7106e-aeac-4954-a862-3b500adb7994 admin admin] The token cache is being invalidate because project 980b2412c0e34520bb0db01eb8f172e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:49.224191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4dc7106e-aeac-4954-a862-3b500adb7994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:49.225187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4dc7106e-aeac-4954-a862-3b500adb7994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:49.230006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 888/1804] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:48 2022] DELETE /identity/v3/projects/980b2412c0e34520bb0db01eb8f172e4 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:30:49.235358 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c0ec0-be69-487c-ad36-efb72ad5d8e8 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.235995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c0ec0-be69-487c-ad36-efb72ad5d8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.236405 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c0ec0-be69-487c-ad36-efb72ad5d8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.270085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-145c0ec0-be69-487c-ad36-efb72ad5d8e8 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:49.271459 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-145c0ec0-be69-487c-ad36-efb72ad5d8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:49.288102 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a4189-e4bf-4360-b9b7-b5a1d7e5af80 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:49.295279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac8544-03b9-48d7-8bb6-475205a44b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:49.295983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac8544-03b9-48d7-8bb6-475205a44b0d admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.296274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac8544-03b9-48d7-8bb6-475205a44b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.296554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac8544-03b9-48d7-8bb6-475205a44b0d admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.297431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac8544-03b9-48d7-8bb6-475205a44b0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:49.299449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac8544-03b9-48d7-8bb6-475205a44b0d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:49.335449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 917/1805] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:49 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:49.336394 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac8544-03b9-48d7-8bb6-475205a44b0d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:49.340023 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7fac8544-03b9-48d7-8bb6-475205a44b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:49.354745 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac8544-03b9-48d7-8bb6-475205a44b0d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:49.360595 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0fc73-b6db-4dbc-ac79-4ed171f4b680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:49.361442 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0fc73-b6db-4dbc-ac79-4ed171f4b680 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.361775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0fc73-b6db-4dbc-ac79-4ed171f4b680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.362008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0fc73-b6db-4dbc-ac79-4ed171f4b680 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.362904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0fc73-b6db-4dbc-ac79-4ed171f4b680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:49.364379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0fc73-b6db-4dbc-ac79-4ed171f4b680 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:49.374316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 889/1806] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:30:49.377596 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 918/1807] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:49.382309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c0ec0-be69-487c-ad36-efb72ad5d8e8 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:49.384500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46454d2-ca7d-40df-a4b7-03ec9a742914 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.384987 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46454d2-ca7d-40df-a4b7-03ec9a742914 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.385390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46454d2-ca7d-40df-a4b7-03ec9a742914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.396211 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c0ec0-be69-487c-ad36-efb72ad5d8e8 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:49.404578 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc16d8-e411-4b4b-bf47-149c631e8cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:49.405556 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc16d8-e411-4b4b-bf47-149c631e8cff admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.406015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc16d8-e411-4b4b-bf47-149c631e8cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.406440 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc16d8-e411-4b4b-bf47-149c631e8cff admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.406919 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-e46454d2-ca7d-40df-a4b7-03ec9a742914 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:49.407522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc16d8-e411-4b4b-bf47-149c631e8cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:49.408331 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e46454d2-ca7d-40df-a4b7-03ec9a742914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:49.409666 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc16d8-e411-4b4b-bf47-149c631e8cff admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:49.418325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 919/1808] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:49.424512 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46454d2-ca7d-40df-a4b7-03ec9a742914 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:49.428646 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 890/1809] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:49.431333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc16d8-e411-4b4b-bf47-149c631e8cff admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:49.439576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46454d2-ca7d-40df-a4b7-03ec9a742914 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:49.449195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3217c-133b-405a-b8f6-0a31d90c0c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:49.449425 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc16d8-e411-4b4b-bf47-149c631e8cff admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:49.449978 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3217c-133b-405a-b8f6-0a31d90c0c3c admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.450272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3217c-133b-405a-b8f6-0a31d90c0c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.450552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3217c-133b-405a-b8f6-0a31d90c0c3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.451293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3217c-133b-405a-b8f6-0a31d90c0c3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:49.451603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3217c-133b-405a-b8f6-0a31d90c0c3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:49.453262 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3217c-133b-405a-b8f6-0a31d90c0c3c admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:49.455130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac865623-b689-4375-8d22-3d5f8a69c0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:49.456385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac865623-b689-4375-8d22-3d5f8a69c0a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.456385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac865623-b689-4375-8d22-3d5f8a69c0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.456587 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac865623-b689-4375-8d22-3d5f8a69c0a4 admin admin] PATH_INFO: `/v3/projects/5963e7eeeac14b4baa08b80580f4ccbf/users/ca287fb1ebcc438490a376dd9e906494/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.469635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:49.471696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 920/1810] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:30:49.472745 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3217c-133b-405a-b8f6-0a31d90c0c3c admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:49.479003 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac865623-b689-4375-8d22-3d5f8a69c0a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5963e7eeeac14b4baa08b80580f4ccbf, user_id=ca287fb1ebcc438490a376dd9e906494, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:49.482561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac865623-b689-4375-8d22-3d5f8a69c0a4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:49.484194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3217c-133b-405a-b8f6-0a31d90c0c3c admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:49.489717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808a895-9f34-4d25-a721-84d2e5fc14bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:49.490508 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808a895-9f34-4d25-a721-84d2e5fc14bc admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.490904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808a895-9f34-4d25-a721-84d2e5fc14bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.491293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808a895-9f34-4d25-a721-84d2e5fc14bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.492033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808a895-9f34-4d25-a721-84d2e5fc14bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:49.495426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808a895-9f34-4d25-a721-84d2e5fc14bc admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:49.518339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 891/1811] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:49 2022] PUT /identity/v3/projects/5963e7eeeac14b4baa08b80580f4ccbf/users/ca287fb1ebcc438490a376dd9e906494/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:49.523619 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5067a-ed3a-403a-9975-db3dd978c88f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.523951 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5067a-ed3a-403a-9975-db3dd978c88f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.524923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5067a-ed3a-403a-9975-db3dd978c88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.536342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 921/1812] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:49 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:49.541432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808a895-9f34-4d25-a721-84d2e5fc14bc admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:49.583560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1cf5067a-ed3a-403a-9975-db3dd978c88f None None] MFA Rules not processed for user `ca287fb1ebcc438490a376dd9e906494`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:49.584844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1cf5067a-ed3a-403a-9975-db3dd978c88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:49.663276 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808a895-9f34-4d25-a721-84d2e5fc14bc admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:49.670508 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30316682-a08b-49a6-9daf-ba32da234466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:49.671218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30316682-a08b-49a6-9daf-ba32da234466 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.671507 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30316682-a08b-49a6-9daf-ba32da234466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.671843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30316682-a08b-49a6-9daf-ba32da234466 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.672561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30316682-a08b-49a6-9daf-ba32da234466 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:49.674579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30316682-a08b-49a6-9daf-ba32da234466 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:49.706462 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 922/1813] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:49 2022] POST /identity/v3/users => generated 283 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:49.707524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30316682-a08b-49a6-9daf-ba32da234466 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:49.707923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 892/1814] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:49.712266 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf5067a-ed3a-403a-9975-db3dd978c88f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:49.729457 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30316682-a08b-49a6-9daf-ba32da234466 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:49.732409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf5067a-ed3a-403a-9975-db3dd978c88f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:49.738055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ee07e-3fd7-4af5-b541-0b1323a1d081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:49.739027 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ee07e-3fd7-4af5-b541-0b1323a1d081 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.739493 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ee07e-3fd7-4af5-b541-0b1323a1d081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.740040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ee07e-3fd7-4af5-b541-0b1323a1d081 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.741053 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ee07e-3fd7-4af5-b541-0b1323a1d081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:49.743903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42029da0-86ac-4c9f-ae91-8018ffe63aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:49.744334 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42029da0-86ac-4c9f-ae91-8018ffe63aec admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.744892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ee07e-3fd7-4af5-b541-0b1323a1d081 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:49.745020 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42029da0-86ac-4c9f-ae91-8018ffe63aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.745765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42029da0-86ac-4c9f-ae91-8018ffe63aec admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.747135 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42029da0-86ac-4c9f-ae91-8018ffe63aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:49.751592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42029da0-86ac-4c9f-ae91-8018ffe63aec admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:49.803992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 893/1815] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:49 2022] POST /identity/v3/projects => generated 381 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:30:49.811522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ee07e-3fd7-4af5-b541-0b1323a1d081 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:49.826183 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 923/1816] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:49.828262 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42029da0-86ac-4c9f-ae91-8018ffe63aec admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:49.843603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:49.853434 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42029da0-86ac-4c9f-ae91-8018ffe63aec admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:49.949658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ee07e-3fd7-4af5-b541-0b1323a1d081 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:49.956914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df24cdb-10ab-4a37-a1ff-5a7858ae294c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:49.958890 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df24cdb-10ab-4a37-a1ff-5a7858ae294c admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.959174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df24cdb-10ab-4a37-a1ff-5a7858ae294c admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.959450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df24cdb-10ab-4a37-a1ff-5a7858ae294c admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.960191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df24cdb-10ab-4a37-a1ff-5a7858ae294c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:49.962609 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df24cdb-10ab-4a37-a1ff-5a7858ae294c admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:49.964497 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d28655-52f0-4761-8cd2-c254b26ec853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:49.965182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d28655-52f0-4761-8cd2-c254b26ec853 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:49.965578 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d28655-52f0-4761-8cd2-c254b26ec853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:49.965869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d28655-52f0-4761-8cd2-c254b26ec853 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:49.966559 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d28655-52f0-4761-8cd2-c254b26ec853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:49.968499 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d28655-52f0-4761-8cd2-c254b26ec853 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:49.985260 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 924/1817] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:30:49.989805 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df24cdb-10ab-4a37-a1ff-5a7858ae294c admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:49.999617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 894/1818] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:49 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:50.002907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d28655-52f0-4761-8cd2-c254b26ec853 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:50.004152 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df24cdb-10ab-4a37-a1ff-5a7858ae294c admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:50.016778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931718af-0131-43b8-ac63-061dce205e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:50.017875 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931718af-0131-43b8-ac63-061dce205e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:50.018164 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931718af-0131-43b8-ac63-061dce205e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:50.018530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931718af-0131-43b8-ac63-061dce205e17 admin admin] PATH_INFO: `/v3/projects/f1d73c7f43cc4d5d9d94563e995e0cdd/users/967979acd963412fa7ec0bfbaacb84e0/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:50.029302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d28655-52f0-4761-8cd2-c254b26ec853 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:50.037617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42aa892-aa31-47f2-a231-adbbb3ca3d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:50.038882 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42aa892-aa31-47f2-a231-adbbb3ca3d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:50.039186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42aa892-aa31-47f2-a231-adbbb3ca3d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:50.039472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42aa892-aa31-47f2-a231-adbbb3ca3d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:50.041590 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42aa892-aa31-47f2-a231-adbbb3ca3d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:50.046604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42aa892-aa31-47f2-a231-adbbb3ca3d57 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:50.047386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931718af-0131-43b8-ac63-061dce205e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d73c7f43cc4d5d9d94563e995e0cdd, user_id=967979acd963412fa7ec0bfbaacb84e0, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:50.052128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931718af-0131-43b8-ac63-061dce205e17 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:50.074491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 895/1819] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:50.075170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42aa892-aa31-47f2-a231-adbbb3ca3d57 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:50.077756 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 925/1820] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:49 2022] PUT /identity/v3/projects/f1d73c7f43cc4d5d9d94563e995e0cdd/users/967979acd963412fa7ec0bfbaacb84e0/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:30:50.087931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42aa892-aa31-47f2-a231-adbbb3ca3d57 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:50.092128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9be561-1fbc-43a0-b722-8f49165edc16 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:50.093537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9be561-1fbc-43a0-b722-8f49165edc16 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:50.094865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9be561-1fbc-43a0-b722-8f49165edc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:50.137238 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:50.159730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-0e9be561-1fbc-43a0-b722-8f49165edc16 None None] MFA Rules not processed for user `967979acd963412fa7ec0bfbaacb84e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:50.162959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0e9be561-1fbc-43a0-b722-8f49165edc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:50.199046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f76010-32e6-43e8-9361-8a76551e8537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:50.202225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f76010-32e6-43e8-9361-8a76551e8537 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:50.202225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f76010-32e6-43e8-9361-8a76551e8537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:50.202225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f76010-32e6-43e8-9361-8a76551e8537 admin admin] PATH_INFO: `/v3/projects/caf03da5958e4262a8d9a0725d4c834b/users/3e42ec4163714044ab644be9298e1a69/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:50.225377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f76010-32e6-43e8-9361-8a76551e8537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caf03da5958e4262a8d9a0725d4c834b, user_id=3e42ec4163714044ab644be9298e1a69, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:50.230220 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f76010-32e6-43e8-9361-8a76551e8537 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:50.264404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 896/1821] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:50 2022] PUT /identity/v3/projects/caf03da5958e4262a8d9a0725d4c834b/users/3e42ec4163714044ab644be9298e1a69/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:50.279874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f76010-32e6-43e8-9361-8a76551e8537 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:50.295385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 926/1822] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:50.301204 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9be561-1fbc-43a0-b722-8f49165edc16 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:50.316403 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9be561-1fbc-43a0-b722-8f49165edc16 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:50.408493 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f76010-32e6-43e8-9361-8a76551e8537 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:50.418537 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5b11a-6749-433c-be4c-8e53e58bc7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:50.419655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5b11a-6749-433c-be4c-8e53e58bc7e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:50.420227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5b11a-6749-433c-be4c-8e53e58bc7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:50.420697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5b11a-6749-433c-be4c-8e53e58bc7e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:50.422151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5b11a-6749-433c-be4c-8e53e58bc7e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:50.424529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5b11a-6749-433c-be4c-8e53e58bc7e5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:50.434253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c2449-6aa1-4bdb-b91a-34a336c2a77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:50.435033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c2449-6aa1-4bdb-b91a-34a336c2a77d admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:50.435326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c2449-6aa1-4bdb-b91a-34a336c2a77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:50.435689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c2449-6aa1-4bdb-b91a-34a336c2a77d admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:50.436476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c2449-6aa1-4bdb-b91a-34a336c2a77d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:50.439164 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c2449-6aa1-4bdb-b91a-34a336c2a77d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:50.450252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 897/1823] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:30:50.451227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5b11a-6749-433c-be4c-8e53e58bc7e5 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:50.455915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:50.466406 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5b11a-6749-433c-be4c-8e53e58bc7e5 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:50.476854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c81fe-8c6a-41f4-acd0-6393e130edda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:50.479029 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c81fe-8c6a-41f4-acd0-6393e130edda admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:50.479577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c81fe-8c6a-41f4-acd0-6393e130edda admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:50.480103 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c81fe-8c6a-41f4-acd0-6393e130edda admin admin] PATH_INFO: `/v3/projects/caf03da5958e4262a8d9a0725d4c834b/users/3e42ec4163714044ab644be9298e1a69/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:50.483271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c81fe-8c6a-41f4-acd0-6393e130edda admin admin] RBAC: Authorizing `identity:create_grant(project_id=caf03da5958e4262a8d9a0725d4c834b, user_id=3e42ec4163714044ab644be9298e1a69, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:50.486149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c81fe-8c6a-41f4-acd0-6393e130edda admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:50.501375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 927/1824] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:50 2022] POST /identity/v3/projects => generated 363 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:50.501375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c2449-6aa1-4bdb-b91a-34a336c2a77d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:50.522997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 898/1825] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:50 2022] PUT /identity/v3/projects/caf03da5958e4262a8d9a0725d4c834b/users/3e42ec4163714044ab644be9298e1a69/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:50.532643 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a72afe-1567-4e69-8011-9cb6d81ab177 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:50.533112 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a72afe-1567-4e69-8011-9cb6d81ab177 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:50.533531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a72afe-1567-4e69-8011-9cb6d81ab177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:50.598998 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-95a72afe-1567-4e69-8011-9cb6d81ab177 None None] MFA Rules not processed for user `3e42ec4163714044ab644be9298e1a69`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:50.600485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-95a72afe-1567-4e69-8011-9cb6d81ab177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:50.636219 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c2449-6aa1-4bdb-b91a-34a336c2a77d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:50.777564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 899/1826] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:30:50.777786 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151e696-83b4-4ffe-a2c7-157afc11e75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:50.778348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151e696-83b4-4ffe-a2c7-157afc11e75e admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:50.778634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151e696-83b4-4ffe-a2c7-157afc11e75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:50.778912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151e696-83b4-4ffe-a2c7-157afc11e75e admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:50.779668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151e696-83b4-4ffe-a2c7-157afc11e75e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:50.781627 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151e696-83b4-4ffe-a2c7-157afc11e75e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:50.785562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a72afe-1567-4e69-8011-9cb6d81ab177 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:50.787824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-95a72afe-1567-4e69-8011-9cb6d81ab177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:50.810509 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 928/1827] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:50 2022] POST /identity/v3/users => generated 288 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:30:50.814215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151e696-83b4-4ffe-a2c7-157afc11e75e admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:50.827420 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151e696-83b4-4ffe-a2c7-157afc11e75e admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:50.833499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bcf217-bce8-493b-9c85-6d5d8c43827a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:50.834226 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcf217-bce8-493b-9c85-6d5d8c43827a admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:50.834530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcf217-bce8-493b-9c85-6d5d8c43827a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:50.834891 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcf217-bce8-493b-9c85-6d5d8c43827a admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:50.835755 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bcf217-bce8-493b-9c85-6d5d8c43827a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:50.837752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bcf217-bce8-493b-9c85-6d5d8c43827a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:50.850269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 929/1828] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:30:50.855475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bcf217-bce8-493b-9c85-6d5d8c43827a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:50.866511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bcf217-bce8-493b-9c85-6d5d8c43827a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:50.872650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f8af3-110a-4b35-b52a-063fe802f72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:50.873589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f8af3-110a-4b35-b52a-063fe802f72c admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:50.874022 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f8af3-110a-4b35-b52a-063fe802f72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:50.874444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f8af3-110a-4b35-b52a-063fe802f72c admin admin] PATH_INFO: `/v3/projects/cd3a96a06fd64b5495a44a7ee0db4153/users/aa8b75f6da1d4310b372bf490402caef/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:50.879606 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:50.894539 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97f8af3-110a-4b35-b52a-063fe802f72c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd3a96a06fd64b5495a44a7ee0db4153, user_id=aa8b75f6da1d4310b372bf490402caef, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:50.905964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97f8af3-110a-4b35-b52a-063fe802f72c admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:50.931466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 930/1829] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:50 2022] PUT /identity/v3/projects/cd3a96a06fd64b5495a44a7ee0db4153/users/aa8b75f6da1d4310b372bf490402caef/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:30:50.931998 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a72afe-1567-4e69-8011-9cb6d81ab177 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:50.939476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f8af3-110a-4b35-b52a-063fe802f72c admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:51.025635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f87a99-b275-4c58-8bbf-900d8dee7766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:51.026727 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f87a99-b275-4c58-8bbf-900d8dee7766 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:51.027204 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f87a99-b275-4c58-8bbf-900d8dee7766 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:51.027807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f87a99-b275-4c58-8bbf-900d8dee7766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:51.057813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:51.073574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f8af3-110a-4b35-b52a-063fe802f72c admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:51.080403 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52797b1-b747-433d-961d-97db127f86a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:51.081981 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52797b1-b747-433d-961d-97db127f86a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:51.082461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52797b1-b747-433d-961d-97db127f86a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:51.083149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52797b1-b747-433d-961d-97db127f86a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:51.084087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52797b1-b747-433d-961d-97db127f86a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:51.086097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52797b1-b747-433d-961d-97db127f86a6 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:51.113349 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 931/1830] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:51.114707 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52797b1-b747-433d-961d-97db127f86a6 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:51.126283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52797b1-b747-433d-961d-97db127f86a6 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:51.135592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bffc3e-a3da-4127-8284-e97f6ccf6710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:51.136817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bffc3e-a3da-4127-8284-e97f6ccf6710 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:51.137329 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bffc3e-a3da-4127-8284-e97f6ccf6710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:51.137795 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bffc3e-a3da-4127-8284-e97f6ccf6710 admin admin] PATH_INFO: `/v3/projects/cd3a96a06fd64b5495a44a7ee0db4153/users/aa8b75f6da1d4310b372bf490402caef/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:51.140174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bffc3e-a3da-4127-8284-e97f6ccf6710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd3a96a06fd64b5495a44a7ee0db4153, user_id=aa8b75f6da1d4310b372bf490402caef, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:51.143743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f87a99-b275-4c58-8bbf-900d8dee7766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:51.144796 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bffc3e-a3da-4127-8284-e97f6ccf6710 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:51.145766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f87a99-b275-4c58-8bbf-900d8dee7766 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:51.158727 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 900/1831] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:51.174176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 932/1832] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:51 2022] PUT /identity/v3/projects/cd3a96a06fd64b5495a44a7ee0db4153/users/aa8b75f6da1d4310b372bf490402caef/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:51.189191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76808e8a-171b-4879-86f2-a8eb64eb9e42 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:51.189191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76808e8a-171b-4879-86f2-a8eb64eb9e42 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:51.189191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76808e8a-171b-4879-86f2-a8eb64eb9e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:51.238413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-76808e8a-171b-4879-86f2-a8eb64eb9e42 None None] MFA Rules not processed for user `aa8b75f6da1d4310b372bf490402caef`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:51.239687 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-76808e8a-171b-4879-86f2-a8eb64eb9e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:51.297243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bffc3e-a3da-4127-8284-e97f6ccf6710 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:51.356282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 901/1833] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:51 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:51.376415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76808e8a-171b-4879-86f2-a8eb64eb9e42 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:51.377970 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-76808e8a-171b-4879-86f2-a8eb64eb9e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:51.387175 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bffc3e-a3da-4127-8284-e97f6ccf6710 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:51.391452 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273936e-afb6-44bb-b6f9-687a10779773 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:51.393304 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273936e-afb6-44bb-b6f9-687a10779773 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:51.393680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273936e-afb6-44bb-b6f9-687a10779773 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:51.394025 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273936e-afb6-44bb-b6f9-687a10779773 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:51.395978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76808e8a-171b-4879-86f2-a8eb64eb9e42 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:51.400165 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac822146-4483-4421-84d8-b697de949915 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:51.400910 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac822146-4483-4421-84d8-b697de949915 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:51.401313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac822146-4483-4421-84d8-b697de949915 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:51.401666 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac822146-4483-4421-84d8-b697de949915 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:51.411974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac822146-4483-4421-84d8-b697de949915 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:51.413467 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac822146-4483-4421-84d8-b697de949915 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:51.445006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 902/1834] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:51 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:30:51.485624 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273936e-afb6-44bb-b6f9-687a10779773 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:51.487856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273936e-afb6-44bb-b6f9-687a10779773 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:51.498963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 933/1835] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:30:51.540468 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac822146-4483-4421-84d8-b697de949915 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:51.545078 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:51.551234 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac822146-4483-4421-84d8-b697de949915 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:51.556230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a66793-1266-4771-9eba-1c1455ba2c59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:51.557004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a66793-1266-4771-9eba-1c1455ba2c59 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:51.557457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a66793-1266-4771-9eba-1c1455ba2c59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:51.558059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a66793-1266-4771-9eba-1c1455ba2c59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:51.622609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a66793-1266-4771-9eba-1c1455ba2c59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:51.624492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a66793-1266-4771-9eba-1c1455ba2c59 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:51.636259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 903/1836] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:51 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:51.863024 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273936e-afb6-44bb-b6f9-687a10779773 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:51.964669 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273936e-afb6-44bb-b6f9-687a10779773 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:51.969469 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bb2c3-6181-488a-b0f6-c95e1ced3d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:51.970461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bb2c3-6181-488a-b0f6-c95e1ced3d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:51.970814 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bb2c3-6181-488a-b0f6-c95e1ced3d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:51.971211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bb2c3-6181-488a-b0f6-c95e1ced3d75 admin admin] PATH_INFO: `/v3/users/ca287fb1ebcc438490a376dd9e906494` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:51.972530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4bb2c3-6181-488a-b0f6-c95e1ced3d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca287fb1ebcc438490a376dd9e906494)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:51.974606 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4bb2c3-6181-488a-b0f6-c95e1ced3d75 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:52.015170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:52.035841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ac4bb2c3-6181-488a-b0f6-c95e1ced3d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca287fb1ebcc438490a376dd9e906494'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:52.048533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ac4bb2c3-6181-488a-b0f6-c95e1ced3d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca287fb1ebcc438490a376dd9e906494'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:52.053180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ac4bb2c3-6181-488a-b0f6-c95e1ced3d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca287fb1ebcc438490a376dd9e906494'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:52.059032 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ac4bb2c3-6181-488a-b0f6-c95e1ced3d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca287fb1ebcc438490a376dd9e906494'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:52.059688 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ac4bb2c3-6181-488a-b0f6-c95e1ced3d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca287fb1ebcc438490a376dd9e906494'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:52.077201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 934/1837] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:51 2022] DELETE /identity/v3/users/ca287fb1ebcc438490a376dd9e906494 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:52.079157 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a661ce-692d-473d-a418-9cba9ae7de3f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:52.079530 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a661ce-692d-473d-a418-9cba9ae7de3f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:52.079819 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a661ce-692d-473d-a418-9cba9ae7de3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:52.099329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-51a661ce-692d-473d-a418-9cba9ae7de3f None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:52.100527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-51a661ce-692d-473d-a418-9cba9ae7de3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:52.186531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 904/1838] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:52.196850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bb2c3-6181-488a-b0f6-c95e1ced3d75 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:52.198323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ac4bb2c3-6181-488a-b0f6-c95e1ced3d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:52.267955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bb2c3-6181-488a-b0f6-c95e1ced3d75 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:52.271698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec3df7-a7bc-4447-9101-fe7f1940f17f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:52.272368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec3df7-a7bc-4447-9101-fe7f1940f17f service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:52.272651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec3df7-a7bc-4447-9101-fe7f1940f17f service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:52.272926 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec3df7-a7bc-4447-9101-fe7f1940f17f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:52.283154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec3df7-a7bc-4447-9101-fe7f1940f17f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:52.284632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec3df7-a7bc-4447-9101-fe7f1940f17f service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:52.297887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 935/1839] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:52.622749 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a661ce-692d-473d-a418-9cba9ae7de3f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:52.624288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-51a661ce-692d-473d-a418-9cba9ae7de3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:52.650748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a661ce-692d-473d-a418-9cba9ae7de3f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:52.671861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451f092-5f2a-42b0-aecd-1aa22c6db63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:52.675153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451f092-5f2a-42b0-aecd-1aa22c6db63b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:52.676122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451f092-5f2a-42b0-aecd-1aa22c6db63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:52.677042 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451f092-5f2a-42b0-aecd-1aa22c6db63b admin admin] PATH_INFO: `/v3/projects/5963e7eeeac14b4baa08b80580f4ccbf` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:52.679909 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451f092-5f2a-42b0-aecd-1aa22c6db63b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5963e7eeeac14b4baa08b80580f4ccbf)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:52.682424 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451f092-5f2a-42b0-aecd-1aa22c6db63b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:52.759017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0451f092-5f2a-42b0-aecd-1aa22c6db63b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5963e7eeeac14b4baa08b80580f4ccbf', 'request_id': 'req-0451f092-5f2a-42b0-aecd-1aa22c6db63b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:52.772607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0451f092-5f2a-42b0-aecd-1aa22c6db63b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5963e7eeeac14b4baa08b80580f4ccbf', 'request_id': 'req-0451f092-5f2a-42b0-aecd-1aa22c6db63b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:52.780676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0451f092-5f2a-42b0-aecd-1aa22c6db63b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5963e7eeeac14b4baa08b80580f4ccbf', 'request_id': 'req-0451f092-5f2a-42b0-aecd-1aa22c6db63b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:52.799679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0451f092-5f2a-42b0-aecd-1aa22c6db63b admin admin] The token cache is being invalidate because project 5963e7eeeac14b4baa08b80580f4ccbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:52.799959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0451f092-5f2a-42b0-aecd-1aa22c6db63b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:52.800670 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0451f092-5f2a-42b0-aecd-1aa22c6db63b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:52.805119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 905/1840] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:52 2022] DELETE /identity/v3/projects/5963e7eeeac14b4baa08b80580f4ccbf => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:52.806448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec3df7-a7bc-4447-9101-fe7f1940f17f service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:52.823008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec3df7-a7bc-4447-9101-fe7f1940f17f service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:52.942867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a88cb8-edbf-4967-b641-7c34710c5cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:52.943772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a88cb8-edbf-4967-b641-7c34710c5cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:52.944054 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a88cb8-edbf-4967-b641-7c34710c5cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:52.944448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a88cb8-edbf-4967-b641-7c34710c5cfc admin admin] PATH_INFO: `/v3/users/3e42ec4163714044ab644be9298e1a69` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:52.945779 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a88cb8-edbf-4967-b641-7c34710c5cfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e42ec4163714044ab644be9298e1a69)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:52.950061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a88cb8-edbf-4967-b641-7c34710c5cfc admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:53.017956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-91a88cb8-edbf-4967-b641-7c34710c5cfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e42ec4163714044ab644be9298e1a69'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:53.033085 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:53.051222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-91a88cb8-edbf-4967-b641-7c34710c5cfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e42ec4163714044ab644be9298e1a69'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:53.052533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-91a88cb8-edbf-4967-b641-7c34710c5cfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e42ec4163714044ab644be9298e1a69'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:53.064493 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-91a88cb8-edbf-4967-b641-7c34710c5cfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e42ec4163714044ab644be9298e1a69'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:53.067357 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-91a88cb8-edbf-4967-b641-7c34710c5cfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e42ec4163714044ab644be9298e1a69'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:53.113577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 936/1841] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:52 2022] DELETE /identity/v3/users/3e42ec4163714044ab644be9298e1a69 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:53.555341 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451f092-5f2a-42b0-aecd-1aa22c6db63b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:53.557873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0451f092-5f2a-42b0-aecd-1aa22c6db63b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:53.586155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451f092-5f2a-42b0-aecd-1aa22c6db63b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:53.625602 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:53.730788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cafb8-0ab1-4fe0-a4e8-735770ad0775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:53.731885 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cafb8-0ab1-4fe0-a4e8-735770ad0775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:53.732291 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cafb8-0ab1-4fe0-a4e8-735770ad0775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:53.732669 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cafb8-0ab1-4fe0-a4e8-735770ad0775 admin admin] PATH_INFO: `/v3/projects/caf03da5958e4262a8d9a0725d4c834b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:53.734660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8cafb8-0ab1-4fe0-a4e8-735770ad0775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=caf03da5958e4262a8d9a0725d4c834b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:53.736705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8cafb8-0ab1-4fe0-a4e8-735770ad0775 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:53.831918 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bb8cafb8-0ab1-4fe0-a4e8-735770ad0775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caf03da5958e4262a8d9a0725d4c834b', 'request_id': 'req-bb8cafb8-0ab1-4fe0-a4e8-735770ad0775'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:53.843189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bb8cafb8-0ab1-4fe0-a4e8-735770ad0775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caf03da5958e4262a8d9a0725d4c834b', 'request_id': 'req-bb8cafb8-0ab1-4fe0-a4e8-735770ad0775'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:53.871226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bb8cafb8-0ab1-4fe0-a4e8-735770ad0775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caf03da5958e4262a8d9a0725d4c834b', 'request_id': 'req-bb8cafb8-0ab1-4fe0-a4e8-735770ad0775'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:53.898861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bb8cafb8-0ab1-4fe0-a4e8-735770ad0775 admin admin] The token cache is being invalidate because project caf03da5958e4262a8d9a0725d4c834b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:53.899396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bb8cafb8-0ab1-4fe0-a4e8-735770ad0775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:53.901434 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bb8cafb8-0ab1-4fe0-a4e8-735770ad0775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:53.903637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 906/1842] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:53 2022] DELETE /identity/v3/projects/caf03da5958e4262a8d9a0725d4c834b => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:53.908983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a916d83-63ac-445d-9709-5ad34d2b79c3 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:53.909508 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a916d83-63ac-445d-9709-5ad34d2b79c3 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:53.911706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a916d83-63ac-445d-9709-5ad34d2b79c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:53.959660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-3a916d83-63ac-445d-9709-5ad34d2b79c3 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:53.960997 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3a916d83-63ac-445d-9709-5ad34d2b79c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:54.079557 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 937/1843] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:54.090671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77d082-6f1c-4658-872f-304d70d24f67 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:54.091114 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77d082-6f1c-4658-872f-304d70d24f67 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:54.092160 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77d082-6f1c-4658-872f-304d70d24f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:54.114503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3e77d082-6f1c-4658-872f-304d70d24f67 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:54.115801 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3e77d082-6f1c-4658-872f-304d70d24f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:54.124813 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 907/1844] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:54.129081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a916d83-63ac-445d-9709-5ad34d2b79c3 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:54.139598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a916d83-63ac-445d-9709-5ad34d2b79c3 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:54.145512 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72118daf-5e42-413e-9d96-e6e47c796e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:54.146235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72118daf-5e42-413e-9d96-e6e47c796e71 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:54.146516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72118daf-5e42-413e-9d96-e6e47c796e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:54.146923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72118daf-5e42-413e-9d96-e6e47c796e71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:54.147767 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72118daf-5e42-413e-9d96-e6e47c796e71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:54.148079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72118daf-5e42-413e-9d96-e6e47c796e71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:54.150675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72118daf-5e42-413e-9d96-e6e47c796e71 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:54.166394 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 938/1845] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:30:54.170997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77d082-6f1c-4658-872f-304d70d24f67 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:54.185434 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77d082-6f1c-4658-872f-304d70d24f67 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:54.190825 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4969d0e-4e8d-476d-959b-802f81953f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:54.191502 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4969d0e-4e8d-476d-959b-802f81953f20 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:54.192087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4969d0e-4e8d-476d-959b-802f81953f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:54.192087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4969d0e-4e8d-476d-959b-802f81953f20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:54.192773 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4969d0e-4e8d-476d-959b-802f81953f20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:54.194768 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4969d0e-4e8d-476d-959b-802f81953f20 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:54.225604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 908/1846] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:54 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:30:54.233052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72118daf-5e42-413e-9d96-e6e47c796e71 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:54.255243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72118daf-5e42-413e-9d96-e6e47c796e71 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:54.325979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e302fc-c3fe-4874-8219-0b20942621b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:54.326764 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e302fc-c3fe-4874-8219-0b20942621b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:54.327451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e302fc-c3fe-4874-8219-0b20942621b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:54.327451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e302fc-c3fe-4874-8219-0b20942621b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:54.328101 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e302fc-c3fe-4874-8219-0b20942621b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:54.330584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e302fc-c3fe-4874-8219-0b20942621b8 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:54.358494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 939/1847] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:54 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:54.359970 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4969d0e-4e8d-476d-959b-802f81953f20 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:54.372747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4969d0e-4e8d-476d-959b-802f81953f20 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:54.378102 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb38bb4-4397-4dfe-bc64-123226d61f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:54.378873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb38bb4-4397-4dfe-bc64-123226d61f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:54.379216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb38bb4-4397-4dfe-bc64-123226d61f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:54.379560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb38bb4-4397-4dfe-bc64-123226d61f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:54.380381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb38bb4-4397-4dfe-bc64-123226d61f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:54.382077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb38bb4-4397-4dfe-bc64-123226d61f4d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:54.394336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 909/1848] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:54.398146 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e302fc-c3fe-4874-8219-0b20942621b8 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:54.408651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e302fc-c3fe-4874-8219-0b20942621b8 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:54.413501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac7dbf-477e-475f-b145-1cf9674c96a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:54.414165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac7dbf-477e-475f-b145-1cf9674c96a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:54.414448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac7dbf-477e-475f-b145-1cf9674c96a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:54.415479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac7dbf-477e-475f-b145-1cf9674c96a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:54.416474 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac7dbf-477e-475f-b145-1cf9674c96a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:54.418111 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac7dbf-477e-475f-b145-1cf9674c96a6 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:54.428671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 940/1849] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:54.433536 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb38bb4-4397-4dfe-bc64-123226d61f4d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:54.443777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb38bb4-4397-4dfe-bc64-123226d61f4d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:54.448929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fb933-d6af-4335-944d-85881f9c1de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:54.449861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fb933-d6af-4335-944d-85881f9c1de9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:54.450142 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fb933-d6af-4335-944d-85881f9c1de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:54.450418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fb933-d6af-4335-944d-85881f9c1de9 admin admin] PATH_INFO: `/v3/projects/043356e0974447e58d6b6654dfc80dff/users/b5f8627aae9747f6b41ce995da380854/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:54.465670 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fb933-d6af-4335-944d-85881f9c1de9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=043356e0974447e58d6b6654dfc80dff, user_id=b5f8627aae9747f6b41ce995da380854, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:54.468322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fb933-d6af-4335-944d-85881f9c1de9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:54.482848 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 910/1850] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:54 2022] PUT /identity/v3/projects/043356e0974447e58d6b6654dfc80dff/users/b5f8627aae9747f6b41ce995da380854/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:30:54.494552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34f961-b06a-4a64-ab65-667b4e78ce1f None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:54.494843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34f961-b06a-4a64-ab65-667b4e78ce1f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:54.495142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34f961-b06a-4a64-ab65-667b4e78ce1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:54.527358 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ea34f961-b06a-4a64-ab65-667b4e78ce1f None None] MFA Rules not processed for user `b5f8627aae9747f6b41ce995da380854`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:54.529224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ea34f961-b06a-4a64-ab65-667b4e78ce1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:54.587869 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:54.639162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 941/1851] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:54.644365 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fb933-d6af-4335-944d-85881f9c1de9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:54.697799 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcdda6-b237-4023-ad33-c24f5ec81365 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:54.698187 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcdda6-b237-4023-ad33-c24f5ec81365 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:54.698558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcdda6-b237-4023-ad33-c24f5ec81365 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:54.700414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 942/1852] 10.222.0.42 () {54 vars in 912 bytes} [Tue Jul 19 15:30:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:30:54.705715 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b691bb-b38c-4bd8-b676-f000c748e024 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:54.706078 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b691bb-b38c-4bd8-b676-f000c748e024 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:54.706449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b691bb-b38c-4bd8-b676-f000c748e024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:54.707770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:54.721769 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fb933-d6af-4335-944d-85881f9c1de9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:54.729968 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-18b691bb-b38c-4bd8-b676-f000c748e024 None None] MFA Rules not processed for user `369bdaded0534142b03ad3cf43a70881`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:54.730366 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf00374-7641-4a75-a1f6-20712d3c0d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:54.731120 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf00374-7641-4a75-a1f6-20712d3c0d88 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:54.731335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-18b691bb-b38c-4bd8-b676-f000c748e024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:54.731474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf00374-7641-4a75-a1f6-20712d3c0d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:54.731889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf00374-7641-4a75-a1f6-20712d3c0d88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:54.732686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf00374-7641-4a75-a1f6-20712d3c0d88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:54.734814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf00374-7641-4a75-a1f6-20712d3c0d88 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:54.788359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 911/1853] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:54 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:30:54.790527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf00374-7641-4a75-a1f6-20712d3c0d88 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:54.806977 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 943/1854] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:30:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:54.820507 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b691bb-b38c-4bd8-b676-f000c748e024 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:54.822936 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-18b691bb-b38c-4bd8-b676-f000c748e024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:54.834044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b691bb-b38c-4bd8-b676-f000c748e024 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:54.859899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf00374-7641-4a75-a1f6-20712d3c0d88 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:54.866390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be29ac-d1c0-4f82-9c34-83fb7bb243dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:54.867176 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be29ac-d1c0-4f82-9c34-83fb7bb243dc admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:54.868152 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be29ac-d1c0-4f82-9c34-83fb7bb243dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:54.868953 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be29ac-d1c0-4f82-9c34-83fb7bb243dc admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:54.872131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be29ac-d1c0-4f82-9c34-83fb7bb243dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:54.872279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be29ac-d1c0-4f82-9c34-83fb7bb243dc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:54.891998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e3d02-fc4a-485b-8b98-094be2537adb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:54.892947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e3d02-fc4a-485b-8b98-094be2537adb service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:54.893498 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e3d02-fc4a-485b-8b98-094be2537adb service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:54.894051 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e3d02-fc4a-485b-8b98-094be2537adb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:54.902037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 912/1855] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:30:54 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:30:54.904139 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be29ac-d1c0-4f82-9c34-83fb7bb243dc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:54.915844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be29ac-d1c0-4f82-9c34-83fb7bb243dc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:54.918504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e3d02-fc4a-485b-8b98-094be2537adb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:54.920076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e3d02-fc4a-485b-8b98-094be2537adb service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:54.922746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dba5eb-270b-4b7a-9782-ff9c1813a78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:54.923434 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dba5eb-270b-4b7a-9782-ff9c1813a78e admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:54.923733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dba5eb-270b-4b7a-9782-ff9c1813a78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:54.924019 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dba5eb-270b-4b7a-9782-ff9c1813a78e admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:54.924765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dba5eb-270b-4b7a-9782-ff9c1813a78e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:54.926415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dba5eb-270b-4b7a-9782-ff9c1813a78e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:54.929852 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 944/1856] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:30:54.939281 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 913/1857] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:30:54.943574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e3d02-fc4a-485b-8b98-094be2537adb service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:54.965777 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e3d02-fc4a-485b-8b98-094be2537adb service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:54.971288 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101c247-bb9e-4df9-88dd-d7e58e2d3a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:54.972189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101c247-bb9e-4df9-88dd-d7e58e2d3a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:54.972472 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101c247-bb9e-4df9-88dd-d7e58e2d3a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:54.972750 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101c247-bb9e-4df9-88dd-d7e58e2d3a8e admin admin] PATH_INFO: `/v3/projects/eb12e67223a6491fa30e1d418b20f416/users/282d845696454f6e86d7dae2748d9026/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:54.989414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101c247-bb9e-4df9-88dd-d7e58e2d3a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb12e67223a6491fa30e1d418b20f416, user_id=282d845696454f6e86d7dae2748d9026, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:54.992429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101c247-bb9e-4df9-88dd-d7e58e2d3a8e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:55.011027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 945/1858] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:54 2022] PUT /identity/v3/projects/eb12e67223a6491fa30e1d418b20f416/users/282d845696454f6e86d7dae2748d9026/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:30:55.014930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dba5eb-270b-4b7a-9782-ff9c1813a78e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:55.033382 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dba5eb-270b-4b7a-9782-ff9c1813a78e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:55.141634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101c247-bb9e-4df9-88dd-d7e58e2d3a8e admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:55.144220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f101c247-bb9e-4df9-88dd-d7e58e2d3a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:55.158243 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d4d3e-c381-4b2f-90aa-27dc1aee06d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:55.158830 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d4d3e-c381-4b2f-90aa-27dc1aee06d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:55.159893 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d4d3e-c381-4b2f-90aa-27dc1aee06d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:55.160203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d4d3e-c381-4b2f-90aa-27dc1aee06d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:55.161287 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d4d3e-c381-4b2f-90aa-27dc1aee06d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:55.163036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d4d3e-c381-4b2f-90aa-27dc1aee06d7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:55.186077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 914/1859] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:30:55.187453 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d4d3e-c381-4b2f-90aa-27dc1aee06d7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:55.212315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d4d3e-c381-4b2f-90aa-27dc1aee06d7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:55.278084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca21f51-335b-4da0-aeaf-5af8a4a2e6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:55.279512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca21f51-335b-4da0-aeaf-5af8a4a2e6e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:55.279803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca21f51-335b-4da0-aeaf-5af8a4a2e6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:55.280083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca21f51-335b-4da0-aeaf-5af8a4a2e6e6 admin admin] PATH_INFO: `/v3/projects/eb12e67223a6491fa30e1d418b20f416/users/282d845696454f6e86d7dae2748d9026/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:55.282433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca21f51-335b-4da0-aeaf-5af8a4a2e6e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb12e67223a6491fa30e1d418b20f416, user_id=282d845696454f6e86d7dae2748d9026, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:55.285337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca21f51-335b-4da0-aeaf-5af8a4a2e6e6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:55.302520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 915/1860] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:30:55 2022] PUT /identity/v3/projects/eb12e67223a6491fa30e1d418b20f416/users/282d845696454f6e86d7dae2748d9026/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:55.314861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbf366-1f5e-4cba-ad14-d1e294aee688 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:55.316028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbf366-1f5e-4cba-ad14-d1e294aee688 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:55.316444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbf366-1f5e-4cba-ad14-d1e294aee688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:55.324541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:55.338004 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101c247-bb9e-4df9-88dd-d7e58e2d3a8e admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:55.351609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-74bbf366-1f5e-4cba-ad14-d1e294aee688 None None] MFA Rules not processed for user `282d845696454f6e86d7dae2748d9026`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:55.353086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-74bbf366-1f5e-4cba-ad14-d1e294aee688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:55.386654 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:55.493175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 916/1861] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:55.493567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d6393-805a-4b1b-a4fd-0f7e8a0038aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:55.495765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d6393-805a-4b1b-a4fd-0f7e8a0038aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:55.496238 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d6393-805a-4b1b-a4fd-0f7e8a0038aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:55.496909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d6393-805a-4b1b-a4fd-0f7e8a0038aa admin admin] PATH_INFO: `/v3/users/967979acd963412fa7ec0bfbaacb84e0` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:55.503150 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d6393-805a-4b1b-a4fd-0f7e8a0038aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=967979acd963412fa7ec0bfbaacb84e0)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:55.505619 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d6393-805a-4b1b-a4fd-0f7e8a0038aa admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:55.510004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbf366-1f5e-4cba-ad14-d1e294aee688 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:55.526166 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-74bbf366-1f5e-4cba-ad14-d1e294aee688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:55.639845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-f91d6393-805a-4b1b-a4fd-0f7e8a0038aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '967979acd963412fa7ec0bfbaacb84e0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:55.661164 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-f91d6393-805a-4b1b-a4fd-0f7e8a0038aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '967979acd963412fa7ec0bfbaacb84e0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:55.662760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-f91d6393-805a-4b1b-a4fd-0f7e8a0038aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '967979acd963412fa7ec0bfbaacb84e0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:55.678009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-f91d6393-805a-4b1b-a4fd-0f7e8a0038aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '967979acd963412fa7ec0bfbaacb84e0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:55.681371 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-f91d6393-805a-4b1b-a4fd-0f7e8a0038aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '967979acd963412fa7ec0bfbaacb84e0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:55.687128 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbf366-1f5e-4cba-ad14-d1e294aee688 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:55.700738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9907801-0691-4104-be88-cfd3650991b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:55.701558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9907801-0691-4104-be88-cfd3650991b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:55.701973 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9907801-0691-4104-be88-cfd3650991b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:55.702310 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9907801-0691-4104-be88-cfd3650991b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:55.706380 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d9907801-0691-4104-be88-cfd3650991b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:55.710546 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 946/1862] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:55 2022] DELETE /identity/v3/users/967979acd963412fa7ec0bfbaacb84e0 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:55.716634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d6393-805a-4b1b-a4fd-0f7e8a0038aa admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:55.720026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f91d6393-805a-4b1b-a4fd-0f7e8a0038aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:55.864941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9907801-0691-4104-be88-cfd3650991b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:55.865418 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d6393-805a-4b1b-a4fd-0f7e8a0038aa admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:55.866854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9907801-0691-4104-be88-cfd3650991b7 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:55.878147 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57012d-c414-488e-950d-1ee692f6f760 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:55.879056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57012d-c414-488e-950d-1ee692f6f760 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:55.879857 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57012d-c414-488e-950d-1ee692f6f760 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:55.880149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57012d-c414-488e-950d-1ee692f6f760 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:55.884410 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1c57012d-c414-488e-950d-1ee692f6f760 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:55.888059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 917/1863] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:30:55.890290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c99a54-5db2-4bf3-bcda-e528bab311f5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:55.890645 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c99a54-5db2-4bf3-bcda-e528bab311f5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:55.890931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c99a54-5db2-4bf3-bcda-e528bab311f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:55.905518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57012d-c414-488e-950d-1ee692f6f760 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:55.908195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57012d-c414-488e-950d-1ee692f6f760 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:55.928984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 947/1864] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:55.932509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-32c99a54-5db2-4bf3-bcda-e528bab311f5 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:55.933800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-32c99a54-5db2-4bf3-bcda-e528bab311f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:56.111465 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 918/1865] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:56.129331 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57012d-c414-488e-950d-1ee692f6f760 service nova] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:56.131630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1c57012d-c414-488e-950d-1ee692f6f760 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:56.159537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:56.280135 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57012d-c414-488e-950d-1ee692f6f760 service nova] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:56.291499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bccae3-4e68-417a-8e55-385350cef3b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:56.294564 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bccae3-4e68-417a-8e55-385350cef3b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:56.294744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bccae3-4e68-417a-8e55-385350cef3b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:56.295517 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bccae3-4e68-417a-8e55-385350cef3b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:56.325335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bccae3-4e68-417a-8e55-385350cef3b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:56.329296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bccae3-4e68-417a-8e55-385350cef3b6 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:56.361101 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 948/1866] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:30:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:30:57.256186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c99a54-5db2-4bf3-bcda-e528bab311f5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:57.259029 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-32c99a54-5db2-4bf3-bcda-e528bab311f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:57.291843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c99a54-5db2-4bf3-bcda-e528bab311f5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:57.310073 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc200330-4e46-4111-b7b4-4058e1e93505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:57.323233 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc200330-4e46-4111-b7b4-4058e1e93505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:57.323518 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc200330-4e46-4111-b7b4-4058e1e93505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:57.328749 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc200330-4e46-4111-b7b4-4058e1e93505 admin admin] PATH_INFO: `/v3/projects/f1d73c7f43cc4d5d9d94563e995e0cdd` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:57.336976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc200330-4e46-4111-b7b4-4058e1e93505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1d73c7f43cc4d5d9d94563e995e0cdd)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:57.348286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc200330-4e46-4111-b7b4-4058e1e93505 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:57.380039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:57.449764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bc200330-4e46-4111-b7b4-4058e1e93505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1d73c7f43cc4d5d9d94563e995e0cdd', 'request_id': 'req-bc200330-4e46-4111-b7b4-4058e1e93505'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:57.465706 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bc200330-4e46-4111-b7b4-4058e1e93505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1d73c7f43cc4d5d9d94563e995e0cdd', 'request_id': 'req-bc200330-4e46-4111-b7b4-4058e1e93505'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:57.506978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bc200330-4e46-4111-b7b4-4058e1e93505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1d73c7f43cc4d5d9d94563e995e0cdd', 'request_id': 'req-bc200330-4e46-4111-b7b4-4058e1e93505'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:57.527804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bc200330-4e46-4111-b7b4-4058e1e93505 admin admin] The token cache is being invalidate because project f1d73c7f43cc4d5d9d94563e995e0cdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:57.528495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bc200330-4e46-4111-b7b4-4058e1e93505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:57.530662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bc200330-4e46-4111-b7b4-4058e1e93505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:57.533612 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 919/1867] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:57 2022] DELETE /identity/v3/projects/f1d73c7f43cc4d5d9d94563e995e0cdd => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:57.542891 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bccae3-4e68-417a-8e55-385350cef3b6 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:57.564924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bccae3-4e68-417a-8e55-385350cef3b6 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:57.645770 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b9f12-49b1-4834-82be-ced17357dd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:57.647363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b9f12-49b1-4834-82be-ced17357dd95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:57.647745 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b9f12-49b1-4834-82be-ced17357dd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:57.648119 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b9f12-49b1-4834-82be-ced17357dd95 admin admin] PATH_INFO: `/v3/users/aa8b75f6da1d4310b372bf490402caef` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:57.651124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b9f12-49b1-4834-82be-ced17357dd95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa8b75f6da1d4310b372bf490402caef)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:57.653337 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b9f12-49b1-4834-82be-ced17357dd95 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:57.726091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c27b9f12-49b1-4834-82be-ced17357dd95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa8b75f6da1d4310b372bf490402caef'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:57.739318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c27b9f12-49b1-4834-82be-ced17357dd95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa8b75f6da1d4310b372bf490402caef'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:57.740121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c27b9f12-49b1-4834-82be-ced17357dd95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa8b75f6da1d4310b372bf490402caef'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:57.751984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c27b9f12-49b1-4834-82be-ced17357dd95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa8b75f6da1d4310b372bf490402caef'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:57.752610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c27b9f12-49b1-4834-82be-ced17357dd95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa8b75f6da1d4310b372bf490402caef'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:57.766305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 949/1868] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:30:57 2022] DELETE /identity/v3/users/aa8b75f6da1d4310b372bf490402caef => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:58.989464 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc200330-4e46-4111-b7b4-4058e1e93505 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:58.990901 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bc200330-4e46-4111-b7b4-4058e1e93505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:59.006294 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc200330-4e46-4111-b7b4-4058e1e93505 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:59.093264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309d108-30e1-4198-ada6-a2081217381f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:59.094766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309d108-30e1-4198-ada6-a2081217381f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:59.094766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309d108-30e1-4198-ada6-a2081217381f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:59.094965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309d108-30e1-4198-ada6-a2081217381f admin admin] PATH_INFO: `/v3/projects/cd3a96a06fd64b5495a44a7ee0db4153` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:59.097807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309d108-30e1-4198-ada6-a2081217381f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd3a96a06fd64b5495a44a7ee0db4153)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:59.100446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309d108-30e1-4198-ada6-a2081217381f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:59.361637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e309d108-30e1-4198-ada6-a2081217381f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd3a96a06fd64b5495a44a7ee0db4153', 'request_id': 'req-e309d108-30e1-4198-ada6-a2081217381f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:59.393327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e309d108-30e1-4198-ada6-a2081217381f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd3a96a06fd64b5495a44a7ee0db4153', 'request_id': 'req-e309d108-30e1-4198-ada6-a2081217381f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:59.418940 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e309d108-30e1-4198-ada6-a2081217381f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd3a96a06fd64b5495a44a7ee0db4153', 'request_id': 'req-e309d108-30e1-4198-ada6-a2081217381f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:59.456608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e309d108-30e1-4198-ada6-a2081217381f admin admin] The token cache is being invalidate because project cd3a96a06fd64b5495a44a7ee0db4153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:30:59.457254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e309d108-30e1-4198-ada6-a2081217381f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:59.458325 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e309d108-30e1-4198-ada6-a2081217381f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:30:59.464287 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 920/1869] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:30:58 2022] DELETE /identity/v3/projects/cd3a96a06fd64b5495a44a7ee0db4153 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:30:59.468714 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf7cac-aea6-401e-8649-72a0c8e754ab None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:59.469273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf7cac-aea6-401e-8649-72a0c8e754ab None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:59.469380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf7cac-aea6-401e-8649-72a0c8e754ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:59.548243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ffaf7cac-aea6-401e-8649-72a0c8e754ab None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:59.549640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ffaf7cac-aea6-401e-8649-72a0c8e754ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:59.611051 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:30:59.744232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 950/1870] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:59.752380 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23112a7-32d9-4987-89f7-1be3a19c059e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:59.752797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23112a7-32d9-4987-89f7-1be3a19c059e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:59.753305 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23112a7-32d9-4987-89f7-1be3a19c059e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:59.813710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c23112a7-32d9-4987-89f7-1be3a19c059e None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:30:59.815080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c23112a7-32d9-4987-89f7-1be3a19c059e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:30:59.833031 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 921/1871] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:30:59.837976 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf7cac-aea6-401e-8649-72a0c8e754ab None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:59.852543 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf7cac-aea6-401e-8649-72a0c8e754ab None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:30:59.889846 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78cacd-c008-4253-aca5-400d92bac13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:30:59.891866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78cacd-c008-4253-aca5-400d92bac13a admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:30:59.893540 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78cacd-c008-4253-aca5-400d92bac13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:30:59.894074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78cacd-c008-4253-aca5-400d92bac13a admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:30:59.896036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78cacd-c008-4253-aca5-400d92bac13a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:30:59.896998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78cacd-c008-4253-aca5-400d92bac13a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:30:59.903940 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78cacd-c008-4253-aca5-400d92bac13a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:30:59.976428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 951/1872] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:30:59.980635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23112a7-32d9-4987-89f7-1be3a19c059e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:30:59.992285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23112a7-32d9-4987-89f7-1be3a19c059e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:00.010543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b378fbc-6587-49f1-9b86-9d772cde740c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:00.011471 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b378fbc-6587-49f1-9b86-9d772cde740c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:00.011827 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b378fbc-6587-49f1-9b86-9d772cde740c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:00.012229 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b378fbc-6587-49f1-9b86-9d772cde740c admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:00.013031 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b378fbc-6587-49f1-9b86-9d772cde740c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:00.015338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b378fbc-6587-49f1-9b86-9d772cde740c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:00.079701 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 922/1873] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:30:59 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:00.089843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78cacd-c008-4253-aca5-400d92bac13a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:00.111317 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78cacd-c008-4253-aca5-400d92bac13a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:00.308787 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e132cc-fe8c-4c6c-8fd8-cc857b892a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:00.309614 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e132cc-fe8c-4c6c-8fd8-cc857b892a29 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:00.309900 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e132cc-fe8c-4c6c-8fd8-cc857b892a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:00.310217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e132cc-fe8c-4c6c-8fd8-cc857b892a29 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:00.311648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e132cc-fe8c-4c6c-8fd8-cc857b892a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:00.313783 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e132cc-fe8c-4c6c-8fd8-cc857b892a29 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:00.345389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b378fbc-6587-49f1-9b86-9d772cde740c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:00.346638 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 952/1874] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:00 2022] POST /identity/v3/users => generated 291 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:00.354429 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:00.364662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b378fbc-6587-49f1-9b86-9d772cde740c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:00.381918 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2edaa-935a-413f-a296-323ea1e97706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:00.382916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2edaa-935a-413f-a296-323ea1e97706 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:00.384099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2edaa-935a-413f-a296-323ea1e97706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:00.384535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2edaa-935a-413f-a296-323ea1e97706 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:00.385575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2edaa-935a-413f-a296-323ea1e97706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:00.389995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2edaa-935a-413f-a296-323ea1e97706 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:00.419868 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e132cc-fe8c-4c6c-8fd8-cc857b892a29 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:00.420116 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 923/1875] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:31:00.433974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e132cc-fe8c-4c6c-8fd8-cc857b892a29 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:00.448033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1ccd9-5d2c-47bb-b1b2-8fbf79468ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:00.448733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1ccd9-5d2c-47bb-b1b2-8fbf79468ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:00.449021 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1ccd9-5d2c-47bb-b1b2-8fbf79468ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:00.449342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1ccd9-5d2c-47bb-b1b2-8fbf79468ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:00.450091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1ccd9-5d2c-47bb-b1b2-8fbf79468ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:00.451936 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1ccd9-5d2c-47bb-b1b2-8fbf79468ff3 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:00.480441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2edaa-935a-413f-a296-323ea1e97706 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:00.480596 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 953/1876] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:00.500964 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2edaa-935a-413f-a296-323ea1e97706 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:00.507894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba27c4-60a0-4ba4-8386-5caf0c6daebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:00.509620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba27c4-60a0-4ba4-8386-5caf0c6daebf admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:00.510124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba27c4-60a0-4ba4-8386-5caf0c6daebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:00.510872 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba27c4-60a0-4ba4-8386-5caf0c6daebf admin admin] PATH_INFO: `/v3/projects/94f3a74478ce44d5bb3c9cb35531f9ed/users/ffd0a4c5e46648b68585df4e1275538d/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:00.572580 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba27c4-60a0-4ba4-8386-5caf0c6daebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f3a74478ce44d5bb3c9cb35531f9ed, user_id=ffd0a4c5e46648b68585df4e1275538d, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:00.582483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba27c4-60a0-4ba4-8386-5caf0c6daebf admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:00.639736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 924/1877] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:00 2022] PUT /identity/v3/projects/94f3a74478ce44d5bb3c9cb35531f9ed/users/ffd0a4c5e46648b68585df4e1275538d/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:00.653415 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63776ff7-d412-42e2-8213-575841d61e34 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:00.653916 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63776ff7-d412-42e2-8213-575841d61e34 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:00.654351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63776ff7-d412-42e2-8213-575841d61e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:00.712068 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-63776ff7-d412-42e2-8213-575841d61e34 None None] MFA Rules not processed for user `ffd0a4c5e46648b68585df4e1275538d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:00.714419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-63776ff7-d412-42e2-8213-575841d61e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:00.817372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 954/1878] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:00.822404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba27c4-60a0-4ba4-8386-5caf0c6daebf admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:00.917309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba27c4-60a0-4ba4-8386-5caf0c6daebf admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:00.922323 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbb911-1c9b-4ba9-909b-15ee00a6c0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:00.923005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbb911-1c9b-4ba9-909b-15ee00a6c0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:00.923291 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbb911-1c9b-4ba9-909b-15ee00a6c0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:00.923569 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbb911-1c9b-4ba9-909b-15ee00a6c0b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:00.924273 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbb911-1c9b-4ba9-909b-15ee00a6c0b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:00.926318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbb911-1c9b-4ba9-909b-15ee00a6c0b0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:00.967372 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 925/1879] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:00 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:00.971738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63776ff7-d412-42e2-8213-575841d61e34 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:00.994783 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63776ff7-d412-42e2-8213-575841d61e34 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:01.025482 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:01.071918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837eeec-14ab-4a7d-8ed4-ae3ce50c2623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:01.072618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837eeec-14ab-4a7d-8ed4-ae3ce50c2623 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:01.073544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837eeec-14ab-4a7d-8ed4-ae3ce50c2623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:01.073544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837eeec-14ab-4a7d-8ed4-ae3ce50c2623 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:01.074083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837eeec-14ab-4a7d-8ed4-ae3ce50c2623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:01.076132 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837eeec-14ab-4a7d-8ed4-ae3ce50c2623 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:01.121253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 955/1880] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:00 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:31:01.129843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbb911-1c9b-4ba9-909b-15ee00a6c0b0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:01.182463 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbb911-1c9b-4ba9-909b-15ee00a6c0b0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:01.199198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69238771-1906-4374-bf58-7c46d0db022a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:01.199895 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69238771-1906-4374-bf58-7c46d0db022a admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:01.200585 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69238771-1906-4374-bf58-7c46d0db022a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:01.200868 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69238771-1906-4374-bf58-7c46d0db022a admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:01.201675 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69238771-1906-4374-bf58-7c46d0db022a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:01.203300 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69238771-1906-4374-bf58-7c46d0db022a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:01.224157 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 926/1881] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:01.224643 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837eeec-14ab-4a7d-8ed4-ae3ce50c2623 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:01.231591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69238771-1906-4374-bf58-7c46d0db022a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:01.234451 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-69238771-1906-4374-bf58-7c46d0db022a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:01.241074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837eeec-14ab-4a7d-8ed4-ae3ce50c2623 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:01.253006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511756ba-486e-484b-9c84-8c1484fcaacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:01.254115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511756ba-486e-484b-9c84-8c1484fcaacf admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:01.254434 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511756ba-486e-484b-9c84-8c1484fcaacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:01.256340 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511756ba-486e-484b-9c84-8c1484fcaacf admin admin] PATH_INFO: `/v3/projects/58dba7c65ee64fafa4454ae795cee430/users/b305db4dec3c41769a8a680b9b220baf/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:01.290750 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511756ba-486e-484b-9c84-8c1484fcaacf admin admin] RBAC: Authorizing `identity:create_grant(project_id=58dba7c65ee64fafa4454ae795cee430, user_id=b305db4dec3c41769a8a680b9b220baf, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:01.298204 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511756ba-486e-484b-9c84-8c1484fcaacf admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:01.328775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 956/1882] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:01 2022] PUT /identity/v3/projects/58dba7c65ee64fafa4454ae795cee430/users/b305db4dec3c41769a8a680b9b220baf/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:31:01.330244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511756ba-486e-484b-9c84-8c1484fcaacf admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:01.367845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69238771-1906-4374-bf58-7c46d0db022a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:01.463301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52ab23-3d7f-47dd-a060-c15f84b26146 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:01.464005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52ab23-3d7f-47dd-a060-c15f84b26146 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:01.464288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52ab23-3d7f-47dd-a060-c15f84b26146 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:01.465281 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52ab23-3d7f-47dd-a060-c15f84b26146 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:01.469388 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9b52ab23-3d7f-47dd-a060-c15f84b26146 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:01.489916 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511756ba-486e-484b-9c84-8c1484fcaacf admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:01.492932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:01.499754 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b2123-fd2b-49dd-8f32-6de2c82f88ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:01.503500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b2123-fd2b-49dd-8f32-6de2c82f88ed admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:01.504435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b2123-fd2b-49dd-8f32-6de2c82f88ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:01.504762 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b2123-fd2b-49dd-8f32-6de2c82f88ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:01.509175 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b2123-fd2b-49dd-8f32-6de2c82f88ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:01.511079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b2123-fd2b-49dd-8f32-6de2c82f88ed admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:01.540279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 957/1883] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:01.542070 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b2123-fd2b-49dd-8f32-6de2c82f88ed admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:01.562080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b2123-fd2b-49dd-8f32-6de2c82f88ed admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:01.576989 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9acfa1e-35b0-47f0-a420-48b1ad6bdda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:01.578282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9acfa1e-35b0-47f0-a420-48b1ad6bdda8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:01.580882 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9acfa1e-35b0-47f0-a420-48b1ad6bdda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:01.581199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9acfa1e-35b0-47f0-a420-48b1ad6bdda8 admin admin] PATH_INFO: `/v3/projects/58dba7c65ee64fafa4454ae795cee430/users/b305db4dec3c41769a8a680b9b220baf/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:01.587086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9acfa1e-35b0-47f0-a420-48b1ad6bdda8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58dba7c65ee64fafa4454ae795cee430, user_id=b305db4dec3c41769a8a680b9b220baf, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:01.598164 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9acfa1e-35b0-47f0-a420-48b1ad6bdda8 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:01.606012 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52ab23-3d7f-47dd-a060-c15f84b26146 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:01.607535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52ab23-3d7f-47dd-a060-c15f84b26146 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:01.631123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 958/1884] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:01 2022] PUT /identity/v3/projects/58dba7c65ee64fafa4454ae795cee430/users/b305db4dec3c41769a8a680b9b220baf/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:01.634853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8cf07-664a-4169-9f93-118fba147ca4 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:01.635361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8cf07-664a-4169-9f93-118fba147ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:01.635800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8cf07-664a-4169-9f93-118fba147ca4 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:01.636645 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 927/1885] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:31:01.637613 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 959/1886] 10.222.0.42 () {54 vars in 921 bytes} [Tue Jul 19 15:31:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:31:01.638453 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f29bb-1c0e-4b1e-bab1-2be2ec843277 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:01.638745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f29bb-1c0e-4b1e-bab1-2be2ec843277 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:01.639024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f29bb-1c0e-4b1e-bab1-2be2ec843277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:01.657478 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21babff-be2c-43de-b832-c3e7d073b6a8 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:01.657912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21babff-be2c-43de-b832-c3e7d073b6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:01.660085 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21babff-be2c-43de-b832-c3e7d073b6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:01.708133 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2c3f29bb-1c0e-4b1e-bab1-2be2ec843277 None None] MFA Rules not processed for user `b305db4dec3c41769a8a680b9b220baf`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:01.709804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2c3f29bb-1c0e-4b1e-bab1-2be2ec843277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:01.717843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-b21babff-be2c-43de-b832-c3e7d073b6a8 None None] MFA Rules not processed for user `bff6a2d4bc734904b2615608a86b097d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:01.721059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b21babff-be2c-43de-b832-c3e7d073b6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:01.872848 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 928/1887] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:01.911197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 960/1888] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Jul 19 15:31:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:31:02.058484 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f29bb-1c0e-4b1e-bab1-2be2ec843277 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:02.060008 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2c3f29bb-1c0e-4b1e-bab1-2be2ec843277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:02.144826 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f29bb-1c0e-4b1e-bab1-2be2ec843277 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:02.159085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b284b-194d-4c32-a30d-3a69a283d582 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:02.164874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b284b-194d-4c32-a30d-3a69a283d582 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:02.166738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b284b-194d-4c32-a30d-3a69a283d582 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:02.167060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b284b-194d-4c32-a30d-3a69a283d582 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:02.184772 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b284b-194d-4c32-a30d-3a69a283d582 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:02.189804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b284b-194d-4c32-a30d-3a69a283d582 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:02.209113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 929/1889] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:02.493841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21babff-be2c-43de-b832-c3e7d073b6a8 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:02.495437 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b21babff-be2c-43de-b832-c3e7d073b6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:02.509929 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:02.527168 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21babff-be2c-43de-b832-c3e7d073b6a8 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:02.543264 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431816aa-662c-4006-bbf7-3379949be1e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:02.545565 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431816aa-662c-4006-bbf7-3379949be1e7 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:02.546122 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431816aa-662c-4006-bbf7-3379949be1e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:02.546715 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431816aa-662c-4006-bbf7-3379949be1e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:02.557367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431816aa-662c-4006-bbf7-3379949be1e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:02.559523 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431816aa-662c-4006-bbf7-3379949be1e7 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:02.573488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 961/1890] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:02.812217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b284b-194d-4c32-a30d-3a69a283d582 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:02.822507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b284b-194d-4c32-a30d-3a69a283d582 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:02.827874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f5bfe-c083-4d7d-99bb-2846d6214276 tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b305db4dec3c41769a8a680b9b220baf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58dba7c65ee64fafa4454ae795cee430', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:02.828546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f5bfe-c083-4d7d-99bb-2846d6214276 tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:02.828835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f5bfe-c083-4d7d-99bb-2846d6214276 tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:02.829118 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f5bfe-c083-4d7d-99bb-2846d6214276 tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:02.829920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f5bfe-c083-4d7d-99bb-2846d6214276 tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:31:02.830205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f5bfe-c083-4d7d-99bb-2846d6214276 tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:02.832252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f5bfe-c083-4d7d-99bb-2846d6214276 tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:02.849857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 930/1891] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:31:02.851609 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431816aa-662c-4006-bbf7-3379949be1e7 service nova] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:02.863544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431816aa-662c-4006-bbf7-3379949be1e7 service nova] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:02.870503 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5dae0-81ec-474c-8c9c-300539bca4c7 tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b305db4dec3c41769a8a680b9b220baf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58dba7c65ee64fafa4454ae795cee430', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:02.871416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5dae0-81ec-474c-8c9c-300539bca4c7 tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:02.871709 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5dae0-81ec-474c-8c9c-300539bca4c7 tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:02.871992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5dae0-81ec-474c-8c9c-300539bca4c7 tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:02.872694 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5dae0-81ec-474c-8c9c-300539bca4c7 tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:02.874718 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5dae0-81ec-474c-8c9c-300539bca4c7 tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:02.905690 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 962/1892] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:02 2022] POST /identity/v3/projects => generated 354 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:02.909848 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f5bfe-c083-4d7d-99bb-2846d6214276 tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:02.920501 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f5bfe-c083-4d7d-99bb-2846d6214276 tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:02.958832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:03.025411 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1397c-1ac1-4cc8-967d-21a4a58b6eec tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b305db4dec3c41769a8a680b9b220baf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58dba7c65ee64fafa4454ae795cee430', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:03.026110 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1397c-1ac1-4cc8-967d-21a4a58b6eec tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:03.026407 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1397c-1ac1-4cc8-967d-21a4a58b6eec tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:03.026691 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1397c-1ac1-4cc8-967d-21a4a58b6eec tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:03.027450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1397c-1ac1-4cc8-967d-21a4a58b6eec tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:31:03.027768 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1397c-1ac1-4cc8-967d-21a4a58b6eec tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:03.029466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1397c-1ac1-4cc8-967d-21a4a58b6eec tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:03.035208 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457908e-b0c8-45fc-8267-7d30d8e538d6 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:03.035516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457908e-b0c8-45fc-8267-7d30d8e538d6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:03.036370 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457908e-b0c8-45fc-8267-7d30d8e538d6 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:03.037485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 963/1893] 10.222.0.42 () {54 vars in 921 bytes} [Tue Jul 19 15:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:31:03.049576 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 931/1894] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:31:03.100462 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a5f8c-20c4-477d-9e5b-fec7cc061b07 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:03.100982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a5f8c-20c4-477d-9e5b-fec7cc061b07 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:03.101455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a5f8c-20c4-477d-9e5b-fec7cc061b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:03.113008 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1397c-1ac1-4cc8-967d-21a4a58b6eec tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:03.136846 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1397c-1ac1-4cc8-967d-21a4a58b6eec tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:03.143976 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-af0a5f8c-20c4-477d-9e5b-fec7cc061b07 None None] MFA Rules not processed for user `bff6a2d4bc734904b2615608a86b097d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:03.146752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-af0a5f8c-20c4-477d-9e5b-fec7cc061b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:03.150966 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b305db4dec3c41769a8a680b9b220baf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58dba7c65ee64fafa4454ae795cee430', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:03.152579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:03.152579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:03.152579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] PATH_INFO: `/v3/projects/ad202d51e1ee459b8fcfcd7425eeabdd` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:03.154900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ad202d51e1ee459b8fcfcd7425eeabdd)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:03.157109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:03.271785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad202d51e1ee459b8fcfcd7425eeabdd', 'request_id': 'req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:03.287494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad202d51e1ee459b8fcfcd7425eeabdd', 'request_id': 'req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:03.300992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad202d51e1ee459b8fcfcd7425eeabdd', 'request_id': 'req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:03.315555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 964/1895] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Jul 19 15:31:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:03.324783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] The token cache is being invalidate because project ad202d51e1ee459b8fcfcd7425eeabdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:03.325459 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:03.326574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:03.328936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 932/1896] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:03 2022] DELETE /identity/v3/projects/ad202d51e1ee459b8fcfcd7425eeabdd => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:03.401284 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a5f8c-20c4-477d-9e5b-fec7cc061b07 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:03.402788 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-af0a5f8c-20c4-477d-9e5b-fec7cc061b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:03.427460 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a5f8c-20c4-477d-9e5b-fec7cc061b07 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:03.676048 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6c69a-b249-46b9-8e9c-df36d95a9b9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:03.676720 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6c69a-b249-46b9-8e9c-df36d95a9b9c service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:03.677006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6c69a-b249-46b9-8e9c-df36d95a9b9c service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:03.677321 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6c69a-b249-46b9-8e9c-df36d95a9b9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:03.679195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f1a6c69a-b249-46b9-8e9c-df36d95a9b9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:03.692879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6c69a-b249-46b9-8e9c-df36d95a9b9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:03.695201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6c69a-b249-46b9-8e9c-df36d95a9b9c service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:03.706994 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 965/1897] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:03.944628 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:03.947494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:04.059088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa0ce1-a826-4621-8718-311ec5fdaf3a tempest-VolumeQuotaClassesTest-110703153 tempest-VolumeQuotaClassesTest-110703153-project-admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:04.072324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fe9dd-dafd-4a98-a904-c10d3dd105d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:04.073279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fe9dd-dafd-4a98-a904-c10d3dd105d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:04.073572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fe9dd-dafd-4a98-a904-c10d3dd105d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:04.073853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fe9dd-dafd-4a98-a904-c10d3dd105d3 admin admin] PATH_INFO: `/v3/users/ffd0a4c5e46648b68585df4e1275538d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:04.077067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fe9dd-dafd-4a98-a904-c10d3dd105d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffd0a4c5e46648b68585df4e1275538d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:04.077535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fe9dd-dafd-4a98-a904-c10d3dd105d3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:04.152586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:04.159101 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0a8fe9dd-dafd-4a98-a904-c10d3dd105d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffd0a4c5e46648b68585df4e1275538d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:04.173425 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0a8fe9dd-dafd-4a98-a904-c10d3dd105d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffd0a4c5e46648b68585df4e1275538d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:04.174091 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0a8fe9dd-dafd-4a98-a904-c10d3dd105d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffd0a4c5e46648b68585df4e1275538d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:04.174707 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0a8fe9dd-dafd-4a98-a904-c10d3dd105d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffd0a4c5e46648b68585df4e1275538d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:04.181811 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0a8fe9dd-dafd-4a98-a904-c10d3dd105d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffd0a4c5e46648b68585df4e1275538d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:04.204035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 933/1898] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:03 2022] DELETE /identity/v3/users/ffd0a4c5e46648b68585df4e1275538d => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:04.205882 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25571121-0a90-43b4-9138-bd6ed6ebda07 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:04.206492 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25571121-0a90-43b4-9138-bd6ed6ebda07 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:04.206838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25571121-0a90-43b4-9138-bd6ed6ebda07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:04.234578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-25571121-0a90-43b4-9138-bd6ed6ebda07 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:04.237235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-25571121-0a90-43b4-9138-bd6ed6ebda07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:04.310757 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:04.339181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 966/1899] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:31:04.349112 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fe9dd-dafd-4a98-a904-c10d3dd105d3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:04.350631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0a8fe9dd-dafd-4a98-a904-c10d3dd105d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:04.437352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fe9dd-dafd-4a98-a904-c10d3dd105d3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:04.441899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a5ccc-4d32-4458-81a4-f6b4f5e81efb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:04.442853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a5ccc-4d32-4458-81a4-f6b4f5e81efb service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:04.443178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a5ccc-4d32-4458-81a4-f6b4f5e81efb service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:04.443904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a5ccc-4d32-4458-81a4-f6b4f5e81efb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:04.458231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a5ccc-4d32-4458-81a4-f6b4f5e81efb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:04.459967 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a5ccc-4d32-4458-81a4-f6b4f5e81efb service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:04.472325 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 934/1900] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:04.790984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25571121-0a90-43b4-9138-bd6ed6ebda07 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:04.793010 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-25571121-0a90-43b4-9138-bd6ed6ebda07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:04.813424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25571121-0a90-43b4-9138-bd6ed6ebda07 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:04.819734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:04.821000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:04.821431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:04.821883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f admin admin] PATH_INFO: `/v3/projects/94f3a74478ce44d5bb3c9cb35531f9ed` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:04.823495 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=94f3a74478ce44d5bb3c9cb35531f9ed)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:04.826123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:04.913913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94f3a74478ce44d5bb3c9cb35531f9ed', 'request_id': 'req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:04.933850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94f3a74478ce44d5bb3c9cb35531f9ed', 'request_id': 'req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:04.955022 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94f3a74478ce44d5bb3c9cb35531f9ed', 'request_id': 'req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:04.973533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f admin admin] The token cache is being invalidate because project 94f3a74478ce44d5bb3c9cb35531f9ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:04.973901 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:04.974748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:04.976468 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 967/1901] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:04 2022] DELETE /identity/v3/projects/94f3a74478ce44d5bb3c9cb35531f9ed => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:04.981001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a5ccc-4d32-4458-81a4-f6b4f5e81efb service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:04.991462 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a5ccc-4d32-4458-81a4-f6b4f5e81efb service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:05.066981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba230ad8-af46-4bf3-b10d-8622e2e39427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:05.067921 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba230ad8-af46-4bf3-b10d-8622e2e39427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:05.068208 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba230ad8-af46-4bf3-b10d-8622e2e39427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:05.068486 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba230ad8-af46-4bf3-b10d-8622e2e39427 admin admin] PATH_INFO: `/v3/users/b305db4dec3c41769a8a680b9b220baf` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:05.069796 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba230ad8-af46-4bf3-b10d-8622e2e39427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b305db4dec3c41769a8a680b9b220baf)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:05.072492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba230ad8-af46-4bf3-b10d-8622e2e39427 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:05.129603 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba230ad8-af46-4bf3-b10d-8622e2e39427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b305db4dec3c41769a8a680b9b220baf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:05.149926 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:05.152119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba230ad8-af46-4bf3-b10d-8622e2e39427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b305db4dec3c41769a8a680b9b220baf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:05.152917 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba230ad8-af46-4bf3-b10d-8622e2e39427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b305db4dec3c41769a8a680b9b220baf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:05.153581 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba230ad8-af46-4bf3-b10d-8622e2e39427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b305db4dec3c41769a8a680b9b220baf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:05.163778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba230ad8-af46-4bf3-b10d-8622e2e39427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b305db4dec3c41769a8a680b9b220baf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:05.183467 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 935/1902] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:04 2022] DELETE /identity/v3/users/b305db4dec3c41769a8a680b9b220baf => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:05.731162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:05.732728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:05.749067 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c009bd-8c0e-4e7a-a574-b60618bcdb9f admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:05.812604 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac1d56-23af-43c5-9681-bff11ccb6033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:05.813514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac1d56-23af-43c5-9681-bff11ccb6033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:05.813801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac1d56-23af-43c5-9681-bff11ccb6033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:05.814081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac1d56-23af-43c5-9681-bff11ccb6033 admin admin] PATH_INFO: `/v3/projects/58dba7c65ee64fafa4454ae795cee430` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:05.815270 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac1d56-23af-43c5-9681-bff11ccb6033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58dba7c65ee64fafa4454ae795cee430)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:05.817224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac1d56-23af-43c5-9681-bff11ccb6033 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:05.875819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dfac1d56-23af-43c5-9681-bff11ccb6033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58dba7c65ee64fafa4454ae795cee430', 'request_id': 'req-dfac1d56-23af-43c5-9681-bff11ccb6033'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:05.888325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dfac1d56-23af-43c5-9681-bff11ccb6033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58dba7c65ee64fafa4454ae795cee430', 'request_id': 'req-dfac1d56-23af-43c5-9681-bff11ccb6033'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:05.909913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dfac1d56-23af-43c5-9681-bff11ccb6033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58dba7c65ee64fafa4454ae795cee430', 'request_id': 'req-dfac1d56-23af-43c5-9681-bff11ccb6033'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:05.926961 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:05.934782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dfac1d56-23af-43c5-9681-bff11ccb6033 admin admin] The token cache is being invalidate because project 58dba7c65ee64fafa4454ae795cee430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:05.935627 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dfac1d56-23af-43c5-9681-bff11ccb6033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:05.938948 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dfac1d56-23af-43c5-9681-bff11ccb6033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:05.941341 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 968/1903] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:05 2022] DELETE /identity/v3/projects/58dba7c65ee64fafa4454ae795cee430 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:05.945973 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0d25a-3382-46fe-8592-2c81ad8dcbec None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:05.946308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0d25a-3382-46fe-8592-2c81ad8dcbec None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:05.946544 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0d25a-3382-46fe-8592-2c81ad8dcbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:05.965850 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-69b0d25a-3382-46fe-8592-2c81ad8dcbec None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:05.967076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-69b0d25a-3382-46fe-8592-2c81ad8dcbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:06.058165 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 936/1904] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:06.065969 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115ad96-a023-421e-b15a-2c51448f7b2d None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:06.066258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115ad96-a023-421e-b15a-2c51448f7b2d None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:06.066565 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115ad96-a023-421e-b15a-2c51448f7b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:06.086835 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-2115ad96-a023-421e-b15a-2c51448f7b2d None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:06.088053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2115ad96-a023-421e-b15a-2c51448f7b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:06.096646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 969/1905] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:06.103040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0d25a-3382-46fe-8592-2c81ad8dcbec None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:06.112938 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0d25a-3382-46fe-8592-2c81ad8dcbec None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:06.117619 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f03fe-28cb-4b0f-9c26-19ca120b4dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:06.118291 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f03fe-28cb-4b0f-9c26-19ca120b4dff admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:06.118575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f03fe-28cb-4b0f-9c26-19ca120b4dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:06.118856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f03fe-28cb-4b0f-9c26-19ca120b4dff admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:06.119589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f03fe-28cb-4b0f-9c26-19ca120b4dff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:31:06.119897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f03fe-28cb-4b0f-9c26-19ca120b4dff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:06.121517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f03fe-28cb-4b0f-9c26-19ca120b4dff admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:06.137492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 937/1906] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:31:06.138727 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115ad96-a023-421e-b15a-2c51448f7b2d None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:06.148303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115ad96-a023-421e-b15a-2c51448f7b2d None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:06.154522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0b703-03a4-44a1-b7b2-10d7d35bfa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:06.155250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0b703-03a4-44a1-b7b2-10d7d35bfa1b admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:06.155608 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0b703-03a4-44a1-b7b2-10d7d35bfa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:06.155964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0b703-03a4-44a1-b7b2-10d7d35bfa1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:06.156724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0b703-03a4-44a1-b7b2-10d7d35bfa1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:06.158685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0b703-03a4-44a1-b7b2-10d7d35bfa1b admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:06.189828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 970/1907] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:06 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:06.194307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f03fe-28cb-4b0f-9c26-19ca120b4dff admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:06.204448 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f03fe-28cb-4b0f-9c26-19ca120b4dff admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:06.264319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728797d-720f-401c-bfb1-6868fc771a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:06.264996 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728797d-720f-401c-bfb1-6868fc771a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:06.265321 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728797d-720f-401c-bfb1-6868fc771a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:06.265600 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728797d-720f-401c-bfb1-6868fc771a5b admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:06.266314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728797d-720f-401c-bfb1-6868fc771a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:06.268275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728797d-720f-401c-bfb1-6868fc771a5b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:06.291004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 938/1908] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:06 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:06.295121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0b703-03a4-44a1-b7b2-10d7d35bfa1b admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:06.305991 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0b703-03a4-44a1-b7b2-10d7d35bfa1b admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:06.323051 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85e0b0-64a0-4827-b2a3-1932d5e46fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:06.323780 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85e0b0-64a0-4827-b2a3-1932d5e46fbc admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:06.324059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85e0b0-64a0-4827-b2a3-1932d5e46fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:06.324347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85e0b0-64a0-4827-b2a3-1932d5e46fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:06.325114 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85e0b0-64a0-4827-b2a3-1932d5e46fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:06.334258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85e0b0-64a0-4827-b2a3-1932d5e46fbc admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:06.360766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 971/1909] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:06.362024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728797d-720f-401c-bfb1-6868fc771a5b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:06.377239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728797d-720f-401c-bfb1-6868fc771a5b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:06.386702 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f561a7f-b5cd-4e8e-9300-bd4155d50a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:06.388267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f561a7f-b5cd-4e8e-9300-bd4155d50a84 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:06.388559 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f561a7f-b5cd-4e8e-9300-bd4155d50a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:06.388843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f561a7f-b5cd-4e8e-9300-bd4155d50a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:06.389954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f561a7f-b5cd-4e8e-9300-bd4155d50a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:06.391759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f561a7f-b5cd-4e8e-9300-bd4155d50a84 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:06.407760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 939/1910] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:06.418575 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85e0b0-64a0-4827-b2a3-1932d5e46fbc admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:06.456481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85e0b0-64a0-4827-b2a3-1932d5e46fbc admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:06.470508 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687dd9b-cfa3-48bb-bd56-4e4e099956ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:06.472111 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687dd9b-cfa3-48bb-bd56-4e4e099956ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:06.472445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687dd9b-cfa3-48bb-bd56-4e4e099956ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:06.472733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687dd9b-cfa3-48bb-bd56-4e4e099956ac admin admin] PATH_INFO: `/v3/projects/b1cdb12b16cc42d9b6b7f38eec608952/users/8ed46f3793a9452085fb7c97b94a8b53/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:06.503888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687dd9b-cfa3-48bb-bd56-4e4e099956ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1cdb12b16cc42d9b6b7f38eec608952, user_id=8ed46f3793a9452085fb7c97b94a8b53, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:06.509740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687dd9b-cfa3-48bb-bd56-4e4e099956ac admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:06.526248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 972/1911] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:06 2022] PUT /identity/v3/projects/b1cdb12b16cc42d9b6b7f38eec608952/users/8ed46f3793a9452085fb7c97b94a8b53/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:06.538088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74f44f-c11a-49c9-9309-77187125ae38 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:06.539883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74f44f-c11a-49c9-9309-77187125ae38 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:06.540301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74f44f-c11a-49c9-9309-77187125ae38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:06.580410 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6f74f44f-c11a-49c9-9309-77187125ae38 None None] MFA Rules not processed for user `8ed46f3793a9452085fb7c97b94a8b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:06.582048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6f74f44f-c11a-49c9-9309-77187125ae38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:06.622979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:06.669364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 940/1912] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:31:06.670818 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687dd9b-cfa3-48bb-bd56-4e4e099956ac admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:06.749901 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687dd9b-cfa3-48bb-bd56-4e4e099956ac admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:06.755411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907eaf7-0121-49aa-bc51-8b5d178e7dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:06.756106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907eaf7-0121-49aa-bc51-8b5d178e7dcf admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:06.756400 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907eaf7-0121-49aa-bc51-8b5d178e7dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:06.756673 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907eaf7-0121-49aa-bc51-8b5d178e7dcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:06.757412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907eaf7-0121-49aa-bc51-8b5d178e7dcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:06.760162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907eaf7-0121-49aa-bc51-8b5d178e7dcf admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:06.789863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 973/1913] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:06 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:06.794661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74f44f-c11a-49c9-9309-77187125ae38 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:06.806333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74f44f-c11a-49c9-9309-77187125ae38 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:06.870115 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209a106-f5e4-495d-aceb-a9288d866b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:06.871041 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209a106-f5e4-495d-aceb-a9288d866b57 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:06.871387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209a106-f5e4-495d-aceb-a9288d866b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:06.871668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209a106-f5e4-495d-aceb-a9288d866b57 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:06.877175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209a106-f5e4-495d-aceb-a9288d866b57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:06.877175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209a106-f5e4-495d-aceb-a9288d866b57 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:06.907503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 941/1914] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:06 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:06.908778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907eaf7-0121-49aa-bc51-8b5d178e7dcf admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:06.921375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907eaf7-0121-49aa-bc51-8b5d178e7dcf admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:06.926802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ea4d5-1f70-4dee-b381-e90cdff41e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:06.927718 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ea4d5-1f70-4dee-b381-e90cdff41e75 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:06.928235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ea4d5-1f70-4dee-b381-e90cdff41e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:06.928561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ea4d5-1f70-4dee-b381-e90cdff41e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:06.929475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ea4d5-1f70-4dee-b381-e90cdff41e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:06.931118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ea4d5-1f70-4dee-b381-e90cdff41e75 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:06.945190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 974/1915] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:06.946946 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209a106-f5e4-495d-aceb-a9288d866b57 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:06.957573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209a106-f5e4-495d-aceb-a9288d866b57 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:06.963383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c5dda-2c51-40ab-80a7-44466f5504e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:06.964063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c5dda-2c51-40ab-80a7-44466f5504e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:06.964352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c5dda-2c51-40ab-80a7-44466f5504e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:06.964634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c5dda-2c51-40ab-80a7-44466f5504e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:06.965428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c5dda-2c51-40ab-80a7-44466f5504e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:06.967036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c5dda-2c51-40ab-80a7-44466f5504e0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:06.978150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 942/1916] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:06.982398 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ea4d5-1f70-4dee-b381-e90cdff41e75 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:06.992689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ea4d5-1f70-4dee-b381-e90cdff41e75 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:06.997279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24cfd5-4ab1-4e5a-b627-81368007f522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:06.998160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24cfd5-4ab1-4e5a-b627-81368007f522 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:06.998443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24cfd5-4ab1-4e5a-b627-81368007f522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:06.998716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24cfd5-4ab1-4e5a-b627-81368007f522 admin admin] PATH_INFO: `/v3/projects/5e00b7bb3b0549f0b0a68b04d37bce23/users/c409228308a041d286f22381a81c406f/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:07.016895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24cfd5-4ab1-4e5a-b627-81368007f522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e00b7bb3b0549f0b0a68b04d37bce23, user_id=c409228308a041d286f22381a81c406f, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:07.020030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24cfd5-4ab1-4e5a-b627-81368007f522 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:07.035876 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 975/1917] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:06 2022] PUT /identity/v3/projects/5e00b7bb3b0549f0b0a68b04d37bce23/users/c409228308a041d286f22381a81c406f/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:07.046663 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205e8a4-5037-446f-b2d9-6440d1c99aa6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:07.046960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205e8a4-5037-446f-b2d9-6440d1c99aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:07.047243 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205e8a4-5037-446f-b2d9-6440d1c99aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:07.080340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2205e8a4-5037-446f-b2d9-6440d1c99aa6 None None] MFA Rules not processed for user `c409228308a041d286f22381a81c406f`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:07.092468 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2205e8a4-5037-446f-b2d9-6440d1c99aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:07.181190 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 943/1918] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:31:07.183810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24cfd5-4ab1-4e5a-b627-81368007f522 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:07.376118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:07.402051 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24cfd5-4ab1-4e5a-b627-81368007f522 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:07.408604 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba435d-f818-41ec-ba27-2f231f122b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:07.409439 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba435d-f818-41ec-ba27-2f231f122b6f admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:07.409861 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba435d-f818-41ec-ba27-2f231f122b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:07.410267 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba435d-f818-41ec-ba27-2f231f122b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:07.411540 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba435d-f818-41ec-ba27-2f231f122b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:07.414812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba435d-f818-41ec-ba27-2f231f122b6f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:07.497358 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 976/1919] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:07 2022] POST /identity/v3/projects => generated 385 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:07.503395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2205e8a4-5037-446f-b2d9-6440d1c99aa6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:07.520766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2205e8a4-5037-446f-b2d9-6440d1c99aa6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:07.610863 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:07.635524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0fc28-f76b-4e34-a78b-d037311443ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:07.636321 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0fc28-f76b-4e34-a78b-d037311443ea admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:07.637164 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0fc28-f76b-4e34-a78b-d037311443ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:07.637553 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0fc28-f76b-4e34-a78b-d037311443ea admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:07.638362 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0fc28-f76b-4e34-a78b-d037311443ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:07.640793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0fc28-f76b-4e34-a78b-d037311443ea admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:07.657290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba435d-f818-41ec-ba27-2f231f122b6f admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:07.659563 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d8ba435d-f818-41ec-ba27-2f231f122b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:07.680735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 944/1920] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:07 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:07.682479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba435d-f818-41ec-ba27-2f231f122b6f admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:07.689597 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffae3f4-f93b-4428-9f4e-3296129b4578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:07.691209 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffae3f4-f93b-4428-9f4e-3296129b4578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:07.691685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffae3f4-f93b-4428-9f4e-3296129b4578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:07.692339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffae3f4-f93b-4428-9f4e-3296129b4578 admin admin] PATH_INFO: `/v3/users/b5f8627aae9747f6b41ce995da380854` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:07.698487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0fc28-f76b-4e34-a78b-d037311443ea admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:07.698787 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffae3f4-f93b-4428-9f4e-3296129b4578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5f8627aae9747f6b41ce995da380854)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:07.703463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffae3f4-f93b-4428-9f4e-3296129b4578 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:07.715088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0fc28-f76b-4e34-a78b-d037311443ea admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:07.721648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba00a05-0eda-4e03-8cca-0f1ca05b439e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:07.722361 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba00a05-0eda-4e03-8cca-0f1ca05b439e admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:07.722647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba00a05-0eda-4e03-8cca-0f1ca05b439e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:07.722926 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba00a05-0eda-4e03-8cca-0f1ca05b439e admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:07.724285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba00a05-0eda-4e03-8cca-0f1ca05b439e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:07.725288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba00a05-0eda-4e03-8cca-0f1ca05b439e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:07.737093 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 945/1921] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:07.741393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba00a05-0eda-4e03-8cca-0f1ca05b439e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:07.751932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba00a05-0eda-4e03-8cca-0f1ca05b439e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:07.756577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a7524-68bc-4abf-9007-e20e1a73c883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:07.757495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a7524-68bc-4abf-9007-e20e1a73c883 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:07.757842 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a7524-68bc-4abf-9007-e20e1a73c883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:07.758673 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-0ffae3f4-f93b-4428-9f4e-3296129b4578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5f8627aae9747f6b41ce995da380854'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:07.758798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a7524-68bc-4abf-9007-e20e1a73c883 admin admin] PATH_INFO: `/v3/projects/6f43ddb788b549e9bc40c26016862bc7/users/ac237939254a4df5aedf088baafe5854/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:07.771191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-0ffae3f4-f93b-4428-9f4e-3296129b4578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5f8627aae9747f6b41ce995da380854'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:07.771920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-0ffae3f4-f93b-4428-9f4e-3296129b4578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5f8627aae9747f6b41ce995da380854'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:07.775030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a7524-68bc-4abf-9007-e20e1a73c883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f43ddb788b549e9bc40c26016862bc7, user_id=ac237939254a4df5aedf088baafe5854, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:07.777834 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a7524-68bc-4abf-9007-e20e1a73c883 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:07.780557 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-0ffae3f4-f93b-4428-9f4e-3296129b4578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5f8627aae9747f6b41ce995da380854'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:07.781334 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-0ffae3f4-f93b-4428-9f4e-3296129b4578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5f8627aae9747f6b41ce995da380854'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:07.794563 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 946/1922] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:07 2022] PUT /identity/v3/projects/6f43ddb788b549e9bc40c26016862bc7/users/ac237939254a4df5aedf088baafe5854/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:31:07.797762 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 977/1923] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:07 2022] DELETE /identity/v3/users/b5f8627aae9747f6b41ce995da380854 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:07.798631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a7524-68bc-4abf-9007-e20e1a73c883 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:07.800439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-940a7524-68bc-4abf-9007-e20e1a73c883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:07.801553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574482e2-8c75-4e28-8761-a905b600da57 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:07.801870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574482e2-8c75-4e28-8761-a905b600da57 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:07.802152 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574482e2-8c75-4e28-8761-a905b600da57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:07.822076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-574482e2-8c75-4e28-8761-a905b600da57 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:07.823301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-574482e2-8c75-4e28-8761-a905b600da57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:07.868580 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a7524-68bc-4abf-9007-e20e1a73c883 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:07.885554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 978/1924] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:31:07.895184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574482e2-8c75-4e28-8761-a905b600da57 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:07.896595 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-574482e2-8c75-4e28-8761-a905b600da57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:07.929473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ee262-0502-4688-90fd-254ce5267da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:07.930874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ee262-0502-4688-90fd-254ce5267da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:07.931136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ee262-0502-4688-90fd-254ce5267da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:07.931413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ee262-0502-4688-90fd-254ce5267da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:07.932221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ee262-0502-4688-90fd-254ce5267da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:07.933927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ee262-0502-4688-90fd-254ce5267da2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:07.944665 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 947/1925] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:31:07.947917 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:07.948719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ee262-0502-4688-90fd-254ce5267da2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:07.959335 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ee262-0502-4688-90fd-254ce5267da2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:07.960019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574482e2-8c75-4e28-8761-a905b600da57 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:07.964210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab4769-cec4-4799-ac8f-678121023075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:07.964796 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0299115-a118-4060-b77a-a474e61b6b95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:07.965086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab4769-cec4-4799-ac8f-678121023075 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:07.965396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab4769-cec4-4799-ac8f-678121023075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:07.965503 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0299115-a118-4060-b77a-a474e61b6b95 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:07.965673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab4769-cec4-4799-ac8f-678121023075 admin admin] PATH_INFO: `/v3/projects/6f43ddb788b549e9bc40c26016862bc7/users/ac237939254a4df5aedf088baafe5854/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:07.965818 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0299115-a118-4060-b77a-a474e61b6b95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:07.966106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0299115-a118-4060-b77a-a474e61b6b95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:07.967642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab4769-cec4-4799-ac8f-678121023075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f43ddb788b549e9bc40c26016862bc7, user_id=ac237939254a4df5aedf088baafe5854, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:07.970296 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab4769-cec4-4799-ac8f-678121023075 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:07.975518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0299115-a118-4060-b77a-a474e61b6b95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:07.977985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0299115-a118-4060-b77a-a474e61b6b95 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:07.981365 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 948/1926] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:07 2022] PUT /identity/v3/projects/6f43ddb788b549e9bc40c26016862bc7/users/ac237939254a4df5aedf088baafe5854/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:31:07.988041 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 979/1927] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:31:07.991919 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fb3e5-4bd0-4cb6-a117-b7c690446657 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:07.992763 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fb3e5-4bd0-4cb6-a117-b7c690446657 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:07.993272 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fb3e5-4bd0-4cb6-a117-b7c690446657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:08.030494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-367fb3e5-4bd0-4cb6-a117-b7c690446657 None None] MFA Rules not processed for user `ac237939254a4df5aedf088baafe5854`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:08.032285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-367fb3e5-4bd0-4cb6-a117-b7c690446657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:08.057534 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:08.122056 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 949/1928] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:08.132912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0299115-a118-4060-b77a-a474e61b6b95 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:08.134528 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b0299115-a118-4060-b77a-a474e61b6b95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:08.210603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0299115-a118-4060-b77a-a474e61b6b95 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:08.294603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c7fa9-b54b-4334-8946-7d91a7ff0636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:08.295554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c7fa9-b54b-4334-8946-7d91a7ff0636 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:08.296015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c7fa9-b54b-4334-8946-7d91a7ff0636 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:08.296463 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c7fa9-b54b-4334-8946-7d91a7ff0636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:08.307557 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c7fa9-b54b-4334-8946-7d91a7ff0636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:08.309096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c7fa9-b54b-4334-8946-7d91a7ff0636 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:08.319323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 980/1929] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:08.596392 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fb3e5-4bd0-4cb6-a117-b7c690446657 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:08.598170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-367fb3e5-4bd0-4cb6-a117-b7c690446657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:08.699751 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fb3e5-4bd0-4cb6-a117-b7c690446657 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:08.703217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167bace-8815-4460-9e3d-ce11cbed78e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:08.704123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167bace-8815-4460-9e3d-ce11cbed78e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:08.704405 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167bace-8815-4460-9e3d-ce11cbed78e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:08.704763 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167bace-8815-4460-9e3d-ce11cbed78e4 admin admin] PATH_INFO: `/v3/projects/043356e0974447e58d6b6654dfc80dff` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:08.706225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167bace-8815-4460-9e3d-ce11cbed78e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=043356e0974447e58d6b6654dfc80dff)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:08.708144 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167bace-8815-4460-9e3d-ce11cbed78e4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:08.785871 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d167bace-8815-4460-9e3d-ce11cbed78e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '043356e0974447e58d6b6654dfc80dff', 'request_id': 'req-d167bace-8815-4460-9e3d-ce11cbed78e4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:08.798755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d167bace-8815-4460-9e3d-ce11cbed78e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '043356e0974447e58d6b6654dfc80dff', 'request_id': 'req-d167bace-8815-4460-9e3d-ce11cbed78e4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:08.807867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d167bace-8815-4460-9e3d-ce11cbed78e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '043356e0974447e58d6b6654dfc80dff', 'request_id': 'req-d167bace-8815-4460-9e3d-ce11cbed78e4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:08.827958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d167bace-8815-4460-9e3d-ce11cbed78e4 admin admin] The token cache is being invalidate because project 043356e0974447e58d6b6654dfc80dff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:08.828484 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d167bace-8815-4460-9e3d-ce11cbed78e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:08.829581 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d167bace-8815-4460-9e3d-ce11cbed78e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:08.832264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 950/1930] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:08 2022] DELETE /identity/v3/projects/043356e0974447e58d6b6654dfc80dff => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:08.836406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c7fa9-b54b-4334-8946-7d91a7ff0636 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:08.858679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c7fa9-b54b-4334-8946-7d91a7ff0636 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:08.956079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b46f8-75c3-4944-83e1-51c7d377a00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:08.957107 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b46f8-75c3-4944-83e1-51c7d377a00a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:08.957516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b46f8-75c3-4944-83e1-51c7d377a00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:08.957891 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b46f8-75c3-4944-83e1-51c7d377a00a admin admin] PATH_INFO: `/v3/users/282d845696454f6e86d7dae2748d9026` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:08.964827 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b46f8-75c3-4944-83e1-51c7d377a00a admin admin] RBAC: Authorizing `identity:delete_user(user_id=282d845696454f6e86d7dae2748d9026)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:08.969399 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b46f8-75c3-4944-83e1-51c7d377a00a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:09.046878 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-861b46f8-75c3-4944-83e1-51c7d377a00a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '282d845696454f6e86d7dae2748d9026'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:09.048710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:09.054237 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167bace-8815-4460-9e3d-ce11cbed78e4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:09.056030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d167bace-8815-4460-9e3d-ce11cbed78e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:09.092419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-861b46f8-75c3-4944-83e1-51c7d377a00a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '282d845696454f6e86d7dae2748d9026'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:09.093475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-861b46f8-75c3-4944-83e1-51c7d377a00a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '282d845696454f6e86d7dae2748d9026'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:09.105242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-861b46f8-75c3-4944-83e1-51c7d377a00a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '282d845696454f6e86d7dae2748d9026'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:09.106325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-861b46f8-75c3-4944-83e1-51c7d377a00a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '282d845696454f6e86d7dae2748d9026'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:09.121134 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:09.130726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 981/1931] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:08 2022] DELETE /identity/v3/users/282d845696454f6e86d7dae2748d9026 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:09.152311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167bace-8815-4460-9e3d-ce11cbed78e4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:09.270623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda27d84-80a4-4eb7-9739-0d285b4047ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:09.271428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda27d84-80a4-4eb7-9739-0d285b4047ca service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:09.271721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda27d84-80a4-4eb7-9739-0d285b4047ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:09.272006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda27d84-80a4-4eb7-9739-0d285b4047ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:09.275909 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bda27d84-80a4-4eb7-9739-0d285b4047ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:09.357538 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda27d84-80a4-4eb7-9739-0d285b4047ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:09.359074 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda27d84-80a4-4eb7-9739-0d285b4047ca service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:09.383695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 951/1932] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:31:09.602893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b46f8-75c3-4944-83e1-51c7d377a00a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:09.604719 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-861b46f8-75c3-4944-83e1-51c7d377a00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:09.762228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b46f8-75c3-4944-83e1-51c7d377a00a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:09.769911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea3ba5-32d0-4d7d-a387-708af4d9e849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:09.770908 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea3ba5-32d0-4d7d-a387-708af4d9e849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:09.771260 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea3ba5-32d0-4d7d-a387-708af4d9e849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:09.771621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea3ba5-32d0-4d7d-a387-708af4d9e849 admin admin] PATH_INFO: `/v3/projects/eb12e67223a6491fa30e1d418b20f416` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:09.772933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea3ba5-32d0-4d7d-a387-708af4d9e849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb12e67223a6491fa30e1d418b20f416)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:09.777123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea3ba5-32d0-4d7d-a387-708af4d9e849 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:09.870747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-06ea3ba5-32d0-4d7d-a387-708af4d9e849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb12e67223a6491fa30e1d418b20f416', 'request_id': 'req-06ea3ba5-32d0-4d7d-a387-708af4d9e849'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:09.877311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-06ea3ba5-32d0-4d7d-a387-708af4d9e849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb12e67223a6491fa30e1d418b20f416', 'request_id': 'req-06ea3ba5-32d0-4d7d-a387-708af4d9e849'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:09.886610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-06ea3ba5-32d0-4d7d-a387-708af4d9e849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb12e67223a6491fa30e1d418b20f416', 'request_id': 'req-06ea3ba5-32d0-4d7d-a387-708af4d9e849'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:09.908206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-06ea3ba5-32d0-4d7d-a387-708af4d9e849 admin admin] The token cache is being invalidate because project eb12e67223a6491fa30e1d418b20f416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:09.909014 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-06ea3ba5-32d0-4d7d-a387-708af4d9e849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:09.910055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-06ea3ba5-32d0-4d7d-a387-708af4d9e849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:09.912054 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 982/1933] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:09 2022] DELETE /identity/v3/projects/eb12e67223a6491fa30e1d418b20f416 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:09.919810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a0d6a-f48b-4605-8aef-2aabf15ca5d1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:09.919810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a0d6a-f48b-4605-8aef-2aabf15ca5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:09.919810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a0d6a-f48b-4605-8aef-2aabf15ca5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:09.985368 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c90a0d6a-f48b-4605-8aef-2aabf15ca5d1 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:09.988306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c90a0d6a-f48b-4605-8aef-2aabf15ca5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:10.073280 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 952/1934] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:10.082014 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2ce86-3520-47d6-b036-c30f33448a89 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:10.082497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2ce86-3520-47d6-b036-c30f33448a89 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:10.082586 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2ce86-3520-47d6-b036-c30f33448a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:10.107124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-3ab2ce86-3520-47d6-b036-c30f33448a89 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:10.108376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3ab2ce86-3520-47d6-b036-c30f33448a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:10.121440 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 983/1935] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:10.124879 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a0d6a-f48b-4605-8aef-2aabf15ca5d1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:10.146566 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a0d6a-f48b-4605-8aef-2aabf15ca5d1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:10.155981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64e5ba-125e-4f4c-8044-40bdaa7fdb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:10.156691 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64e5ba-125e-4f4c-8044-40bdaa7fdb61 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:10.156994 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64e5ba-125e-4f4c-8044-40bdaa7fdb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:10.157347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64e5ba-125e-4f4c-8044-40bdaa7fdb61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:10.158094 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64e5ba-125e-4f4c-8044-40bdaa7fdb61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:31:10.158386 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64e5ba-125e-4f4c-8044-40bdaa7fdb61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:10.160147 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64e5ba-125e-4f4c-8044-40bdaa7fdb61 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:10.174659 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 953/1936] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:31:10.179012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2ce86-3520-47d6-b036-c30f33448a89 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:10.189791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2ce86-3520-47d6-b036-c30f33448a89 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:10.196063 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c696ca3-d76b-4359-be91-b9b6ad49feff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:10.197581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c696ca3-d76b-4359-be91-b9b6ad49feff admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:10.197944 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c696ca3-d76b-4359-be91-b9b6ad49feff admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:10.198301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c696ca3-d76b-4359-be91-b9b6ad49feff admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:10.199028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c696ca3-d76b-4359-be91-b9b6ad49feff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:10.200950 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c696ca3-d76b-4359-be91-b9b6ad49feff admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:10.239976 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 984/1937] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:10 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:10.244483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64e5ba-125e-4f4c-8044-40bdaa7fdb61 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:10.256941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64e5ba-125e-4f4c-8044-40bdaa7fdb61 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:10.314708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:10.335158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0be77-0413-4e0e-9c7b-038a4f6c2ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:10.336447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0be77-0413-4e0e-9c7b-038a4f6c2ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:10.336835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0be77-0413-4e0e-9c7b-038a4f6c2ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:10.337200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0be77-0413-4e0e-9c7b-038a4f6c2ff4 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:10.338015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0be77-0413-4e0e-9c7b-038a4f6c2ff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:10.339969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0be77-0413-4e0e-9c7b-038a4f6c2ff4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:10.363432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 954/1938] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:10 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:10.367405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c696ca3-d76b-4359-be91-b9b6ad49feff admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:10.380706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c696ca3-d76b-4359-be91-b9b6ad49feff admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:10.389043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a7268-8a12-4c94-96de-cabfad08d200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:10.390124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a7268-8a12-4c94-96de-cabfad08d200 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:10.390530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a7268-8a12-4c94-96de-cabfad08d200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:10.390812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a7268-8a12-4c94-96de-cabfad08d200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:10.391719 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a7268-8a12-4c94-96de-cabfad08d200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:10.393379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a7268-8a12-4c94-96de-cabfad08d200 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:10.407434 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 985/1939] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:10.411124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0be77-0413-4e0e-9c7b-038a4f6c2ff4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:10.423532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0be77-0413-4e0e-9c7b-038a4f6c2ff4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:10.429615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e0d03-ac59-49b8-b6b5-11dbce3d49e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:10.431494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e0d03-ac59-49b8-b6b5-11dbce3d49e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:10.431494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e0d03-ac59-49b8-b6b5-11dbce3d49e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:10.431668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e0d03-ac59-49b8-b6b5-11dbce3d49e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:10.432668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e0d03-ac59-49b8-b6b5-11dbce3d49e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:10.434561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e0d03-ac59-49b8-b6b5-11dbce3d49e0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:10.450303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 955/1940] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:10.451480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a7268-8a12-4c94-96de-cabfad08d200 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:10.464758 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a7268-8a12-4c94-96de-cabfad08d200 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:10.468698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78412e67-9c68-493a-bf9c-561f22be87ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:10.469622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78412e67-9c68-493a-bf9c-561f22be87ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:10.469904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78412e67-9c68-493a-bf9c-561f22be87ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:10.470182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78412e67-9c68-493a-bf9c-561f22be87ff admin admin] PATH_INFO: `/v3/projects/819c3c65b5de48679d24a8a2191a05d0/users/d7d32874d5154484aef4fe4118ba6299/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:10.483905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:10.488307 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78412e67-9c68-493a-bf9c-561f22be87ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=819c3c65b5de48679d24a8a2191a05d0, user_id=d7d32874d5154484aef4fe4118ba6299, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:10.491787 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78412e67-9c68-493a-bf9c-561f22be87ff admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:10.510564 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 986/1941] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:10 2022] PUT /identity/v3/projects/819c3c65b5de48679d24a8a2191a05d0/users/d7d32874d5154484aef4fe4118ba6299/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:10.522039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4dae3-dffb-43ed-93d3-8c63f1317e02 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:10.522333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4dae3-dffb-43ed-93d3-8c63f1317e02 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:10.522620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4dae3-dffb-43ed-93d3-8c63f1317e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:10.552918 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-79b4dae3-dffb-43ed-93d3-8c63f1317e02 None None] MFA Rules not processed for user `d7d32874d5154484aef4fe4118ba6299`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:10.554305 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-79b4dae3-dffb-43ed-93d3-8c63f1317e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:10.625365 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 956/1942] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:31:10.629951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78412e67-9c68-493a-bf9c-561f22be87ff admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:10.704295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78412e67-9c68-493a-bf9c-561f22be87ff admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:10.710105 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f0f16-97dd-4a01-a427-ccc41bd0fafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:10.711060 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f0f16-97dd-4a01-a427-ccc41bd0fafc admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:10.711578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f0f16-97dd-4a01-a427-ccc41bd0fafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:10.711959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f0f16-97dd-4a01-a427-ccc41bd0fafc admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:10.712806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f0f16-97dd-4a01-a427-ccc41bd0fafc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:10.714900 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f0f16-97dd-4a01-a427-ccc41bd0fafc admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:10.762680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 987/1943] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:10 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:10.766426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4dae3-dffb-43ed-93d3-8c63f1317e02 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:10.778275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4dae3-dffb-43ed-93d3-8c63f1317e02 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:10.843042 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ba135-c9e7-4cc6-8128-f8bf9821061d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:10.843732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ba135-c9e7-4cc6-8128-f8bf9821061d admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:10.844023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ba135-c9e7-4cc6-8128-f8bf9821061d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:10.844308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ba135-c9e7-4cc6-8128-f8bf9821061d admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:10.845055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ba135-c9e7-4cc6-8128-f8bf9821061d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:10.847064 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ba135-c9e7-4cc6-8128-f8bf9821061d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:10.868540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 957/1944] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:10 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:10.872548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f0f16-97dd-4a01-a427-ccc41bd0fafc admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:10.885251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f0f16-97dd-4a01-a427-ccc41bd0fafc admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:10.890452 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87c1f5-6e42-43f8-acad-988d67b86a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:10.891497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87c1f5-6e42-43f8-acad-988d67b86a8c admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:10.892060 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87c1f5-6e42-43f8-acad-988d67b86a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:10.892406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87c1f5-6e42-43f8-acad-988d67b86a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:10.893186 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87c1f5-6e42-43f8-acad-988d67b86a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:10.894881 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87c1f5-6e42-43f8-acad-988d67b86a8c admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:10.908557 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 988/1945] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:10.912728 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ba135-c9e7-4cc6-8128-f8bf9821061d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:10.936316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ba135-c9e7-4cc6-8128-f8bf9821061d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:10.942924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c280b2c-3746-486f-bd03-6db405c99d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:10.943939 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c280b2c-3746-486f-bd03-6db405c99d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:10.944328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c280b2c-3746-486f-bd03-6db405c99d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:10.944702 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c280b2c-3746-486f-bd03-6db405c99d58 admin admin] PATH_INFO: `/v3/projects/5d4795fd011146c8a77dd1126082b782/users/732289e41d0a4a0f850d12f59994c2df/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:10.964688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c280b2c-3746-486f-bd03-6db405c99d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d4795fd011146c8a77dd1126082b782, user_id=732289e41d0a4a0f850d12f59994c2df, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:10.967605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c280b2c-3746-486f-bd03-6db405c99d58 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:10.994118 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 958/1946] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:10 2022] PUT /identity/v3/projects/5d4795fd011146c8a77dd1126082b782/users/732289e41d0a4a0f850d12f59994c2df/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:10.995596 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87c1f5-6e42-43f8-acad-988d67b86a8c admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:11.008687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87c1f5-6e42-43f8-acad-988d67b86a8c admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:11.116031 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd3736-ae30-4f21-8153-5cd734027d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:11.118996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd3736-ae30-4f21-8153-5cd734027d23 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:11.119503 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd3736-ae30-4f21-8153-5cd734027d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:11.120007 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd3736-ae30-4f21-8153-5cd734027d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:11.120932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd3736-ae30-4f21-8153-5cd734027d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:11.123556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd3736-ae30-4f21-8153-5cd734027d23 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:11.143572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 989/1947] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:11.147751 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c280b2c-3746-486f-bd03-6db405c99d58 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:11.160142 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c280b2c-3746-486f-bd03-6db405c99d58 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:11.166104 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771f83c-4064-4033-9324-e46d2bdc1628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:11.167009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771f83c-4064-4033-9324-e46d2bdc1628 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:11.167463 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771f83c-4064-4033-9324-e46d2bdc1628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:11.167747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771f83c-4064-4033-9324-e46d2bdc1628 admin admin] PATH_INFO: `/v3/projects/5d4795fd011146c8a77dd1126082b782/users/732289e41d0a4a0f850d12f59994c2df/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:11.169933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771f83c-4064-4033-9324-e46d2bdc1628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d4795fd011146c8a77dd1126082b782, user_id=732289e41d0a4a0f850d12f59994c2df, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:11.172700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771f83c-4064-4033-9324-e46d2bdc1628 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:11.184191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 959/1948] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:11 2022] PUT /identity/v3/projects/5d4795fd011146c8a77dd1126082b782/users/732289e41d0a4a0f850d12f59994c2df/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:31:11.196099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ff221-9678-4a6e-8310-d78ad7f4a9a1 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:11.196388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ff221-9678-4a6e-8310-d78ad7f4a9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:11.196668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ff221-9678-4a6e-8310-d78ad7f4a9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:11.224812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-da9ff221-9678-4a6e-8310-d78ad7f4a9a1 None None] MFA Rules not processed for user `732289e41d0a4a0f850d12f59994c2df`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:11.226066 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-da9ff221-9678-4a6e-8310-d78ad7f4a9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:11.280457 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:11.302557 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 990/1949] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:11.314872 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771f83c-4064-4033-9324-e46d2bdc1628 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:11.316567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4771f83c-4064-4033-9324-e46d2bdc1628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:11.322707 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:11.377940 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771f83c-4064-4033-9324-e46d2bdc1628 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:11.381671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b06a1-ac98-425a-b650-69c617f5d34c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:11.382332 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b06a1-ac98-425a-b650-69c617f5d34c service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:11.383226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b06a1-ac98-425a-b650-69c617f5d34c service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:11.383633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b06a1-ac98-425a-b650-69c617f5d34c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:11.441898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b06a1-ac98-425a-b650-69c617f5d34c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:11.443443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b06a1-ac98-425a-b650-69c617f5d34c service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:11.456496 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 960/1950] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:17.272067 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ff221-9678-4a6e-8310-d78ad7f4a9a1 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:17.365263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ff221-9678-4a6e-8310-d78ad7f4a9a1 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:17.370278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27a32f-9c61-4905-8d4d-9a8799bfed45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:17.371153 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27a32f-9c61-4905-8d4d-9a8799bfed45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:17.371470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27a32f-9c61-4905-8d4d-9a8799bfed45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:17.371760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27a32f-9c61-4905-8d4d-9a8799bfed45 admin admin] PATH_INFO: `/v3/users/d7d32874d5154484aef4fe4118ba6299` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:17.372925 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb27a32f-9c61-4905-8d4d-9a8799bfed45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7d32874d5154484aef4fe4118ba6299)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:17.374873 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb27a32f-9c61-4905-8d4d-9a8799bfed45 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:17.434986 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-eb27a32f-9c61-4905-8d4d-9a8799bfed45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7d32874d5154484aef4fe4118ba6299'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:17.447299 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-eb27a32f-9c61-4905-8d4d-9a8799bfed45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7d32874d5154484aef4fe4118ba6299'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:17.448051 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-eb27a32f-9c61-4905-8d4d-9a8799bfed45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7d32874d5154484aef4fe4118ba6299'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:17.455765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-eb27a32f-9c61-4905-8d4d-9a8799bfed45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7d32874d5154484aef4fe4118ba6299'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:17.456485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-eb27a32f-9c61-4905-8d4d-9a8799bfed45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7d32874d5154484aef4fe4118ba6299'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:17.471255 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 991/1951] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:17 2022] DELETE /identity/v3/users/d7d32874d5154484aef4fe4118ba6299 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:17.475794 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc48084-44e4-4326-ad52-76073e44f7ae None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:17.476993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc48084-44e4-4326-ad52-76073e44f7ae None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:17.477401 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc48084-44e4-4326-ad52-76073e44f7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:17.497877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3bc48084-44e4-4326-ad52-76073e44f7ae None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:17.499119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3bc48084-44e4-4326-ad52-76073e44f7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:17.576369 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 961/1952] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:17.588907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27a32f-9c61-4905-8d4d-9a8799bfed45 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:17.590501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-eb27a32f-9c61-4905-8d4d-9a8799bfed45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:17.704221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27a32f-9c61-4905-8d4d-9a8799bfed45 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:17.704221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74e54d-6e0c-48a4-b683-1e7e2dead4e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:17.704221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74e54d-6e0c-48a4-b683-1e7e2dead4e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:17.704578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74e54d-6e0c-48a4-b683-1e7e2dead4e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:17.704578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74e54d-6e0c-48a4-b683-1e7e2dead4e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:17.719051 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74e54d-6e0c-48a4-b683-1e7e2dead4e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:17.720981 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74e54d-6e0c-48a4-b683-1e7e2dead4e0 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:17.756574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 992/1953] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:31:18.225696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc48084-44e4-4326-ad52-76073e44f7ae None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:18.228180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3bc48084-44e4-4326-ad52-76073e44f7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:18.253466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc48084-44e4-4326-ad52-76073e44f7ae None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:18.261595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:18.262537 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:18.263229 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:18.263567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8 admin admin] PATH_INFO: `/v3/projects/819c3c65b5de48679d24a8a2191a05d0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:18.264831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=819c3c65b5de48679d24a8a2191a05d0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:18.267072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:18.322690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '819c3c65b5de48679d24a8a2191a05d0', 'request_id': 'req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:18.325316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:18.330600 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '819c3c65b5de48679d24a8a2191a05d0', 'request_id': 'req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:18.337799 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '819c3c65b5de48679d24a8a2191a05d0', 'request_id': 'req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:18.348776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8 admin admin] The token cache is being invalidate because project 819c3c65b5de48679d24a8a2191a05d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:18.349067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:18.349734 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:18.351900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 962/1954] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:18 2022] DELETE /identity/v3/projects/819c3c65b5de48679d24a8a2191a05d0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:18.358258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74e54d-6e0c-48a4-b683-1e7e2dead4e0 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:18.368233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74e54d-6e0c-48a4-b683-1e7e2dead4e0 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:18.373272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:18.435396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb9cea-6852-4f4a-a581-1d75925eb41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:18.436203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb9cea-6852-4f4a-a581-1d75925eb41f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:18.436489 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb9cea-6852-4f4a-a581-1d75925eb41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:18.436773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb9cea-6852-4f4a-a581-1d75925eb41f admin admin] PATH_INFO: `/v3/users/732289e41d0a4a0f850d12f59994c2df` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:18.438012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb9cea-6852-4f4a-a581-1d75925eb41f admin admin] RBAC: Authorizing `identity:delete_user(user_id=732289e41d0a4a0f850d12f59994c2df)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:18.439951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb9cea-6852-4f4a-a581-1d75925eb41f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:18.489443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-edeb9cea-6852-4f4a-a581-1d75925eb41f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '732289e41d0a4a0f850d12f59994c2df'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:18.501589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-edeb9cea-6852-4f4a-a581-1d75925eb41f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '732289e41d0a4a0f850d12f59994c2df'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:18.502242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-edeb9cea-6852-4f4a-a581-1d75925eb41f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '732289e41d0a4a0f850d12f59994c2df'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:18.509494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-edeb9cea-6852-4f4a-a581-1d75925eb41f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '732289e41d0a4a0f850d12f59994c2df'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:18.510181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-edeb9cea-6852-4f4a-a581-1d75925eb41f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '732289e41d0a4a0f850d12f59994c2df'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:18.523077 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 993/1955] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:18 2022] DELETE /identity/v3/users/732289e41d0a4a0f850d12f59994c2df => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:18.877625 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:18.880736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:18.906866 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ccb5b-4096-4851-aa2f-a215b4fc6ae8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:18.987105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14679b42-94b3-4d8e-9be4-4203fc8ac7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:18.988257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14679b42-94b3-4d8e-9be4-4203fc8ac7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:18.988542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14679b42-94b3-4d8e-9be4-4203fc8ac7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:18.988814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14679b42-94b3-4d8e-9be4-4203fc8ac7c3 admin admin] PATH_INFO: `/v3/projects/5d4795fd011146c8a77dd1126082b782` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:18.990064 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14679b42-94b3-4d8e-9be4-4203fc8ac7c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d4795fd011146c8a77dd1126082b782)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:18.992023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14679b42-94b3-4d8e-9be4-4203fc8ac7c3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:19.052196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-14679b42-94b3-4d8e-9be4-4203fc8ac7c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d4795fd011146c8a77dd1126082b782', 'request_id': 'req-14679b42-94b3-4d8e-9be4-4203fc8ac7c3'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:19.061692 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-14679b42-94b3-4d8e-9be4-4203fc8ac7c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d4795fd011146c8a77dd1126082b782', 'request_id': 'req-14679b42-94b3-4d8e-9be4-4203fc8ac7c3'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:19.065015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb9cea-6852-4f4a-a581-1d75925eb41f admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:19.066567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-edeb9cea-6852-4f4a-a581-1d75925eb41f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:19.068659 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-14679b42-94b3-4d8e-9be4-4203fc8ac7c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d4795fd011146c8a77dd1126082b782', 'request_id': 'req-14679b42-94b3-4d8e-9be4-4203fc8ac7c3'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:19.087146 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-14679b42-94b3-4d8e-9be4-4203fc8ac7c3 admin admin] The token cache is being invalidate because project 5d4795fd011146c8a77dd1126082b782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:19.087435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-14679b42-94b3-4d8e-9be4-4203fc8ac7c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:19.088108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-14679b42-94b3-4d8e-9be4-4203fc8ac7c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:19.089806 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 963/1956] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:18 2022] DELETE /identity/v3/projects/5d4795fd011146c8a77dd1126082b782 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:19.094844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41ece5-30cb-46db-9537-47cf84544b56 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:19.095213 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41ece5-30cb-46db-9537-47cf84544b56 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:19.095567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41ece5-30cb-46db-9537-47cf84544b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:19.117854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bc41ece5-30cb-46db-9537-47cf84544b56 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:19.119413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bc41ece5-30cb-46db-9537-47cf84544b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:19.146402 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb9cea-6852-4f4a-a581-1d75925eb41f admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:19.203404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:19.228864 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df243854-758c-465f-b5b0-ec4aacade8b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:19.231167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df243854-758c-465f-b5b0-ec4aacade8b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:19.231551 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df243854-758c-465f-b5b0-ec4aacade8b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:19.231828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df243854-758c-465f-b5b0-ec4aacade8b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:19.233350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 964/1957] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:31:19.239932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-df243854-758c-465f-b5b0-ec4aacade8b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:19.247084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23b12e-9d8e-4981-a3b9-067e6f1d9ec5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:19.247403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23b12e-9d8e-4981-a3b9-067e6f1d9ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:19.247653 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23b12e-9d8e-4981-a3b9-067e6f1d9ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:19.274591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ca23b12e-9d8e-4981-a3b9-067e6f1d9ec5 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:19.274591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ca23b12e-9d8e-4981-a3b9-067e6f1d9ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:19.294284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 965/1958] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:19.295375 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23b12e-9d8e-4981-a3b9-067e6f1d9ec5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:19.301044 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:19.313161 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23b12e-9d8e-4981-a3b9-067e6f1d9ec5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:19.319192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae834b-259e-41eb-9eaa-cf93232b144c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:19.319192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae834b-259e-41eb-9eaa-cf93232b144c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:19.319549 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae834b-259e-41eb-9eaa-cf93232b144c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:19.319823 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae834b-259e-41eb-9eaa-cf93232b144c admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:19.320588 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae834b-259e-41eb-9eaa-cf93232b144c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:31:19.320863 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae834b-259e-41eb-9eaa-cf93232b144c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:19.322741 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae834b-259e-41eb-9eaa-cf93232b144c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:19.337282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 966/1959] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:31:19.341938 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae834b-259e-41eb-9eaa-cf93232b144c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:19.347318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df243854-758c-465f-b5b0-ec4aacade8b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:19.349274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df243854-758c-465f-b5b0-ec4aacade8b5 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:19.353296 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae834b-259e-41eb-9eaa-cf93232b144c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:19.358419 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26b25e-9b09-4926-9e45-87eb7f3c9017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:19.359090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26b25e-9b09-4926-9e45-87eb7f3c9017 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:19.359379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26b25e-9b09-4926-9e45-87eb7f3c9017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:19.359662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26b25e-9b09-4926-9e45-87eb7f3c9017 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:19.360366 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26b25e-9b09-4926-9e45-87eb7f3c9017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:19.361923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 994/1960] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:31:19.366138 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26b25e-9b09-4926-9e45-87eb7f3c9017 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:19.419586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 967/1961] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:19 2022] POST /identity/v3/projects => generated 375 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:19.424361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df243854-758c-465f-b5b0-ec4aacade8b5 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:19.446770 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df243854-758c-465f-b5b0-ec4aacade8b5 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:19.550634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8072177-0894-4ac1-bb82-1b5e352a51e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:19.551993 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8072177-0894-4ac1-bb82-1b5e352a51e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:19.552498 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8072177-0894-4ac1-bb82-1b5e352a51e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:19.553079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8072177-0894-4ac1-bb82-1b5e352a51e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:19.554278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8072177-0894-4ac1-bb82-1b5e352a51e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:19.557073 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8072177-0894-4ac1-bb82-1b5e352a51e4 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:19.597582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 995/1962] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:19 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:19.602327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26b25e-9b09-4926-9e45-87eb7f3c9017 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:19.616369 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26b25e-9b09-4926-9e45-87eb7f3c9017 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:19.627475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0e028-a273-41dd-a955-e4eb319a8f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:19.629543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0e028-a273-41dd-a955-e4eb319a8f81 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:19.632856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0e028-a273-41dd-a955-e4eb319a8f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:19.633495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0e028-a273-41dd-a955-e4eb319a8f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:19.635618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0e028-a273-41dd-a955-e4eb319a8f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:19.637408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0e028-a273-41dd-a955-e4eb319a8f81 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:19.655414 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 968/1963] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:19.659046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8072177-0894-4ac1-bb82-1b5e352a51e4 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:19.672584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8072177-0894-4ac1-bb82-1b5e352a51e4 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:19.683118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bcdd3-7541-4471-b3dd-d47bdcdf59fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:19.684833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bcdd3-7541-4471-b3dd-d47bdcdf59fc admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:19.685473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bcdd3-7541-4471-b3dd-d47bdcdf59fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:19.686169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bcdd3-7541-4471-b3dd-d47bdcdf59fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:19.688060 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bcdd3-7541-4471-b3dd-d47bdcdf59fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:19.690385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bcdd3-7541-4471-b3dd-d47bdcdf59fc admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:19.710991 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 996/1964] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:19.712509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0e028-a273-41dd-a955-e4eb319a8f81 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:19.724254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0e028-a273-41dd-a955-e4eb319a8f81 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:19.732503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd779d5d-95c8-4bb8-be7b-2c42ea38a7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:19.734059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd779d5d-95c8-4bb8-be7b-2c42ea38a7a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:19.734458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd779d5d-95c8-4bb8-be7b-2c42ea38a7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:19.735104 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd779d5d-95c8-4bb8-be7b-2c42ea38a7a7 admin admin] PATH_INFO: `/v3/projects/cbe88e8efc054e9da5ac6b6790cb3a66/users/c51857e9cf5c461789bae58821e7b7a5/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:19.753711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd779d5d-95c8-4bb8-be7b-2c42ea38a7a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbe88e8efc054e9da5ac6b6790cb3a66, user_id=c51857e9cf5c461789bae58821e7b7a5, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:19.756476 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd779d5d-95c8-4bb8-be7b-2c42ea38a7a7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:19.773705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 969/1965] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:19 2022] PUT /identity/v3/projects/cbe88e8efc054e9da5ac6b6790cb3a66/users/c51857e9cf5c461789bae58821e7b7a5/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:19.784302 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9e6fa-2dfa-408c-8073-f8501c29ab97 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:19.784595 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9e6fa-2dfa-408c-8073-f8501c29ab97 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:19.784874 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9e6fa-2dfa-408c-8073-f8501c29ab97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:19.813665 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-1ab9e6fa-2dfa-408c-8073-f8501c29ab97 None None] MFA Rules not processed for user `c51857e9cf5c461789bae58821e7b7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:19.818913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1ab9e6fa-2dfa-408c-8073-f8501c29ab97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:19.896364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 997/1966] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:19.910793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd779d5d-95c8-4bb8-be7b-2c42ea38a7a7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:19.913126 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fd779d5d-95c8-4bb8-be7b-2c42ea38a7a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:20.051676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd779d5d-95c8-4bb8-be7b-2c42ea38a7a7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:20.056134 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c693785-1256-48c7-a9fd-53b009e077f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:20.057185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c693785-1256-48c7-a9fd-53b009e077f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:20.057574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c693785-1256-48c7-a9fd-53b009e077f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:20.057858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c693785-1256-48c7-a9fd-53b009e077f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:20.072552 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c693785-1256-48c7-a9fd-53b009e077f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:20.074207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c693785-1256-48c7-a9fd-53b009e077f5 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:20.085959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 970/1967] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:22.031040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9e6fa-2dfa-408c-8073-f8501c29ab97 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:22.032604 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1ab9e6fa-2dfa-408c-8073-f8501c29ab97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:22.044193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:22.157077 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9e6fa-2dfa-408c-8073-f8501c29ab97 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:22.163226 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8335a9-a07f-4cc7-94cd-a1ffea8b5a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:22.165300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8335a9-a07f-4cc7-94cd-a1ffea8b5a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:22.165745 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8335a9-a07f-4cc7-94cd-a1ffea8b5a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:22.166200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8335a9-a07f-4cc7-94cd-a1ffea8b5a26 admin admin] PATH_INFO: `/v3/users/8ed46f3793a9452085fb7c97b94a8b53` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:22.168044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8335a9-a07f-4cc7-94cd-a1ffea8b5a26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ed46f3793a9452085fb7c97b94a8b53)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:22.170444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8335a9-a07f-4cc7-94cd-a1ffea8b5a26 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:22.248724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9c8335a9-a07f-4cc7-94cd-a1ffea8b5a26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ed46f3793a9452085fb7c97b94a8b53'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:22.270824 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9c8335a9-a07f-4cc7-94cd-a1ffea8b5a26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ed46f3793a9452085fb7c97b94a8b53'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:22.271567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9c8335a9-a07f-4cc7-94cd-a1ffea8b5a26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ed46f3793a9452085fb7c97b94a8b53'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:22.282397 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9c8335a9-a07f-4cc7-94cd-a1ffea8b5a26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ed46f3793a9452085fb7c97b94a8b53'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:22.283041 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9c8335a9-a07f-4cc7-94cd-a1ffea8b5a26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ed46f3793a9452085fb7c97b94a8b53'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:22.312703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 998/1968] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:22 2022] DELETE /identity/v3/users/8ed46f3793a9452085fb7c97b94a8b53 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:22.316566 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd98589-85f4-4859-a7f3-fd92b50b1e92 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:22.317406 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd98589-85f4-4859-a7f3-fd92b50b1e92 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:22.317826 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd98589-85f4-4859-a7f3-fd92b50b1e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:22.342968 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0cd98589-85f4-4859-a7f3-fd92b50b1e92 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:22.344203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0cd98589-85f4-4859-a7f3-fd92b50b1e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:22.437979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 971/1969] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:22.457212 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8335a9-a07f-4cc7-94cd-a1ffea8b5a26 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:22.458769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9c8335a9-a07f-4cc7-94cd-a1ffea8b5a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:22.592092 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8335a9-a07f-4cc7-94cd-a1ffea8b5a26 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:22.596802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1afeda9-b80e-4f60-909e-2248dec43c35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:22.604373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1afeda9-b80e-4f60-909e-2248dec43c35 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:22.604373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1afeda9-b80e-4f60-909e-2248dec43c35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:22.604373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1afeda9-b80e-4f60-909e-2248dec43c35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:22.624041 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1afeda9-b80e-4f60-909e-2248dec43c35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:22.625554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1afeda9-b80e-4f60-909e-2248dec43c35 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:22.642397 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 999/1970] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:23.008237 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd98589-85f4-4859-a7f3-fd92b50b1e92 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:23.010262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0cd98589-85f4-4859-a7f3-fd92b50b1e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:23.029420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd98589-85f4-4859-a7f3-fd92b50b1e92 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:23.037822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:23.038856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:23.039169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:23.039465 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c admin admin] PATH_INFO: `/v3/projects/b1cdb12b16cc42d9b6b7f38eec608952` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:23.040744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1cdb12b16cc42d9b6b7f38eec608952)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:23.042722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:23.084281 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:23.105793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1cdb12b16cc42d9b6b7f38eec608952', 'request_id': 'req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:23.115032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1cdb12b16cc42d9b6b7f38eec608952', 'request_id': 'req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:23.123255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1cdb12b16cc42d9b6b7f38eec608952', 'request_id': 'req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:23.137267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c admin admin] The token cache is being invalidate because project b1cdb12b16cc42d9b6b7f38eec608952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:23.137563 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:23.138219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:23.139780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 972/1971] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:23 2022] DELETE /identity/v3/projects/b1cdb12b16cc42d9b6b7f38eec608952 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:23.143767 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1afeda9-b80e-4f60-909e-2248dec43c35 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:23.153791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1afeda9-b80e-4f60-909e-2248dec43c35 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:23.184722 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:23.218541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613e08c-98ab-4ab7-a933-cdde1865daf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:23.219416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613e08c-98ab-4ab7-a933-cdde1865daf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:23.219697 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613e08c-98ab-4ab7-a933-cdde1865daf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:23.219971 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613e08c-98ab-4ab7-a933-cdde1865daf9 admin admin] PATH_INFO: `/v3/users/c409228308a041d286f22381a81c406f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:23.221194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613e08c-98ab-4ab7-a933-cdde1865daf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c409228308a041d286f22381a81c406f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:23.223128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613e08c-98ab-4ab7-a933-cdde1865daf9 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:23.279092 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-2613e08c-98ab-4ab7-a933-cdde1865daf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c409228308a041d286f22381a81c406f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:23.291368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-2613e08c-98ab-4ab7-a933-cdde1865daf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c409228308a041d286f22381a81c406f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:23.292075 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-2613e08c-98ab-4ab7-a933-cdde1865daf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c409228308a041d286f22381a81c406f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:23.299344 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-2613e08c-98ab-4ab7-a933-cdde1865daf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c409228308a041d286f22381a81c406f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:23.300095 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-2613e08c-98ab-4ab7-a933-cdde1865daf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c409228308a041d286f22381a81c406f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:23.313127 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1000/1972] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:23 2022] DELETE /identity/v3/users/c409228308a041d286f22381a81c406f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:23.792390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:23.794156 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:23.820975 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a5d09-a7f0-469d-9517-00bdf1eb873c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:23.889187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d65e75-4349-47a5-82bd-4d1aa43e9030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:23.889999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d65e75-4349-47a5-82bd-4d1aa43e9030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:23.890282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d65e75-4349-47a5-82bd-4d1aa43e9030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:23.890557 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d65e75-4349-47a5-82bd-4d1aa43e9030 admin admin] PATH_INFO: `/v3/projects/5e00b7bb3b0549f0b0a68b04d37bce23` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:23.891782 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d65e75-4349-47a5-82bd-4d1aa43e9030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e00b7bb3b0549f0b0a68b04d37bce23)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:23.894213 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d65e75-4349-47a5-82bd-4d1aa43e9030 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:23.947152 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-66d65e75-4349-47a5-82bd-4d1aa43e9030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e00b7bb3b0549f0b0a68b04d37bce23', 'request_id': 'req-66d65e75-4349-47a5-82bd-4d1aa43e9030'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:23.955262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-66d65e75-4349-47a5-82bd-4d1aa43e9030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e00b7bb3b0549f0b0a68b04d37bce23', 'request_id': 'req-66d65e75-4349-47a5-82bd-4d1aa43e9030'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:23.962226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-66d65e75-4349-47a5-82bd-4d1aa43e9030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e00b7bb3b0549f0b0a68b04d37bce23', 'request_id': 'req-66d65e75-4349-47a5-82bd-4d1aa43e9030'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:23.975312 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-66d65e75-4349-47a5-82bd-4d1aa43e9030 admin admin] The token cache is being invalidate because project 5e00b7bb3b0549f0b0a68b04d37bce23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:23.975704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-66d65e75-4349-47a5-82bd-4d1aa43e9030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:23.976521 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-66d65e75-4349-47a5-82bd-4d1aa43e9030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:23.978463 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 973/1973] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:23 2022] DELETE /identity/v3/projects/5e00b7bb3b0549f0b0a68b04d37bce23 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:23.982750 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613e08c-98ab-4ab7-a933-cdde1865daf9 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:23.984266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2613e08c-98ab-4ab7-a933-cdde1865daf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:24.003057 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613e08c-98ab-4ab7-a933-cdde1865daf9 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:24.063655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ecbcd-53e2-4ecd-8849-f1bd507eeea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:24.064520 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ecbcd-53e2-4ecd-8849-f1bd507eeea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:24.064801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ecbcd-53e2-4ecd-8849-f1bd507eeea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:24.065078 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ecbcd-53e2-4ecd-8849-f1bd507eeea2 admin admin] PATH_INFO: `/v3/users/ac237939254a4df5aedf088baafe5854` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:24.066289 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ecbcd-53e2-4ecd-8849-f1bd507eeea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac237939254a4df5aedf088baafe5854)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:24.068206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ecbcd-53e2-4ecd-8849-f1bd507eeea2 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:24.112625 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-160ecbcd-53e2-4ecd-8849-f1bd507eeea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac237939254a4df5aedf088baafe5854'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:24.114490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:24.127368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-160ecbcd-53e2-4ecd-8849-f1bd507eeea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac237939254a4df5aedf088baafe5854'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:24.128046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-160ecbcd-53e2-4ecd-8849-f1bd507eeea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac237939254a4df5aedf088baafe5854'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:24.135874 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-160ecbcd-53e2-4ecd-8849-f1bd507eeea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac237939254a4df5aedf088baafe5854'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:24.136557 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-160ecbcd-53e2-4ecd-8849-f1bd507eeea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac237939254a4df5aedf088baafe5854'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:24.148693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1001/1974] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:23 2022] DELETE /identity/v3/users/ac237939254a4df5aedf088baafe5854 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:24.421294 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d65e75-4349-47a5-82bd-4d1aa43e9030 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:24.422775 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-66d65e75-4349-47a5-82bd-4d1aa43e9030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:24.439161 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d65e75-4349-47a5-82bd-4d1aa43e9030 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:24.488938 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:24.500854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4575f4-4109-406d-9b57-a175ca4db374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:24.501767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4575f4-4109-406d-9b57-a175ca4db374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:24.502049 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4575f4-4109-406d-9b57-a175ca4db374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:24.502328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4575f4-4109-406d-9b57-a175ca4db374 admin admin] PATH_INFO: `/v3/projects/6f43ddb788b549e9bc40c26016862bc7` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:24.503508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4575f4-4109-406d-9b57-a175ca4db374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f43ddb788b549e9bc40c26016862bc7)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:24.505468 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4575f4-4109-406d-9b57-a175ca4db374 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:24.554609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ab4575f4-4109-406d-9b57-a175ca4db374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f43ddb788b549e9bc40c26016862bc7', 'request_id': 'req-ab4575f4-4109-406d-9b57-a175ca4db374'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:24.562075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ab4575f4-4109-406d-9b57-a175ca4db374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f43ddb788b549e9bc40c26016862bc7', 'request_id': 'req-ab4575f4-4109-406d-9b57-a175ca4db374'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:24.567689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ab4575f4-4109-406d-9b57-a175ca4db374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f43ddb788b549e9bc40c26016862bc7', 'request_id': 'req-ab4575f4-4109-406d-9b57-a175ca4db374'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:24.578702 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ab4575f4-4109-406d-9b57-a175ca4db374 admin admin] The token cache is being invalidate because project 6f43ddb788b549e9bc40c26016862bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:24.578985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ab4575f4-4109-406d-9b57-a175ca4db374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:24.579606 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ab4575f4-4109-406d-9b57-a175ca4db374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:24.581226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 974/1975] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:24 2022] DELETE /identity/v3/projects/6f43ddb788b549e9bc40c26016862bc7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:24.587437 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25f895-3c47-4eba-9c2f-522430efb5df None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:24.587820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25f895-3c47-4eba-9c2f-522430efb5df None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:24.588099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25f895-3c47-4eba-9c2f-522430efb5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:24.606352 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-0e25f895-3c47-4eba-9c2f-522430efb5df None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:24.607549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0e25f895-3c47-4eba-9c2f-522430efb5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:24.667758 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1002/1976] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:24.675775 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667adc7-4ad3-4703-8a58-f8b04b1f15cb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:24.676063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667adc7-4ad3-4703-8a58-f8b04b1f15cb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:24.676338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667adc7-4ad3-4703-8a58-f8b04b1f15cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:24.695584 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5667adc7-4ad3-4703-8a58-f8b04b1f15cb None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:24.696792 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5667adc7-4ad3-4703-8a58-f8b04b1f15cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:24.704642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 975/1977] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:24.708563 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25f895-3c47-4eba-9c2f-522430efb5df None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:24.718272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25f895-3c47-4eba-9c2f-522430efb5df None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:24.722812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7dae5-2e7b-44bf-b69e-19f5e421614d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:24.723472 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7dae5-2e7b-44bf-b69e-19f5e421614d admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:24.723757 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7dae5-2e7b-44bf-b69e-19f5e421614d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:24.724036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7dae5-2e7b-44bf-b69e-19f5e421614d admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:24.724769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7dae5-2e7b-44bf-b69e-19f5e421614d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:31:24.725081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7dae5-2e7b-44bf-b69e-19f5e421614d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:24.727252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7dae5-2e7b-44bf-b69e-19f5e421614d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:24.740722 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1003/1978] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:31:24.745845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667adc7-4ad3-4703-8a58-f8b04b1f15cb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:24.755205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667adc7-4ad3-4703-8a58-f8b04b1f15cb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:24.760211 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3423c-5198-40bb-8e3a-be62c4caa7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:24.760870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3423c-5198-40bb-8e3a-be62c4caa7bc admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:24.761179 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3423c-5198-40bb-8e3a-be62c4caa7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:24.761465 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3423c-5198-40bb-8e3a-be62c4caa7bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:24.762230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3423c-5198-40bb-8e3a-be62c4caa7bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:24.764123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3423c-5198-40bb-8e3a-be62c4caa7bc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:24.792310 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 976/1979] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:24 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:24.800523 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7dae5-2e7b-44bf-b69e-19f5e421614d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:24.810828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7dae5-2e7b-44bf-b69e-19f5e421614d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:24.879280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca5d49-7cf1-4a51-9594-927bdc9c7a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:24.879972 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca5d49-7cf1-4a51-9594-927bdc9c7a35 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:24.880256 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca5d49-7cf1-4a51-9594-927bdc9c7a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:24.880640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca5d49-7cf1-4a51-9594-927bdc9c7a35 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:24.881284 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca5d49-7cf1-4a51-9594-927bdc9c7a35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:24.883737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca5d49-7cf1-4a51-9594-927bdc9c7a35 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:24.906596 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1004/1980] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:24 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:24.910466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3423c-5198-40bb-8e3a-be62c4caa7bc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:24.921809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3423c-5198-40bb-8e3a-be62c4caa7bc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:24.926289 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2fc29-f3dd-47c6-912f-61f8eeffb44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:24.926945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2fc29-f3dd-47c6-912f-61f8eeffb44b admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:24.927227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2fc29-f3dd-47c6-912f-61f8eeffb44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:24.927499 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2fc29-f3dd-47c6-912f-61f8eeffb44b admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:24.928226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2fc29-f3dd-47c6-912f-61f8eeffb44b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:24.930022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2fc29-f3dd-47c6-912f-61f8eeffb44b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:24.940843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 977/1981] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:24.945372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca5d49-7cf1-4a51-9594-927bdc9c7a35 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:24.955427 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca5d49-7cf1-4a51-9594-927bdc9c7a35 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:24.959982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a6dea-872d-49ce-a511-554bcc3ee66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:24.961696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a6dea-872d-49ce-a511-554bcc3ee66e admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:24.961979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a6dea-872d-49ce-a511-554bcc3ee66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:24.962260 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a6dea-872d-49ce-a511-554bcc3ee66e admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:24.962992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a6dea-872d-49ce-a511-554bcc3ee66e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:24.964614 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a6dea-872d-49ce-a511-554bcc3ee66e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:24.974493 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1005/1982] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:24.979196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2fc29-f3dd-47c6-912f-61f8eeffb44b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:24.989058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2fc29-f3dd-47c6-912f-61f8eeffb44b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:24.993711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae5605-b5b3-47b9-a559-a1207651d9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:24.994588 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae5605-b5b3-47b9-a559-a1207651d9f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:24.994871 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae5605-b5b3-47b9-a559-a1207651d9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:24.995147 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae5605-b5b3-47b9-a559-a1207651d9f6 admin admin] PATH_INFO: `/v3/projects/12aebc5f34a54d6699a2efc6d3a1715b/users/9e54c22586b74506b3246eab8de57807/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:25.010772 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae5605-b5b3-47b9-a559-a1207651d9f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12aebc5f34a54d6699a2efc6d3a1715b, user_id=9e54c22586b74506b3246eab8de57807, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:25.013460 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae5605-b5b3-47b9-a559-a1207651d9f6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:25.029087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 978/1983] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:24 2022] PUT /identity/v3/projects/12aebc5f34a54d6699a2efc6d3a1715b/users/9e54c22586b74506b3246eab8de57807/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:31:25.034049 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5262a-03fa-4fee-baaf-68c47780de27 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:25.034340 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5262a-03fa-4fee-baaf-68c47780de27 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:25.034618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5262a-03fa-4fee-baaf-68c47780de27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:25.053841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-25c5262a-03fa-4fee-baaf-68c47780de27 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:25.055160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-25c5262a-03fa-4fee-baaf-68c47780de27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:25.085051 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:25.131573 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1006/1984] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:25.142332 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae5605-b5b3-47b9-a559-a1207651d9f6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:25.144130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-02ae5605-b5b3-47b9-a559-a1207651d9f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:25.203026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae5605-b5b3-47b9-a559-a1207651d9f6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:25.207217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4772a-c6b0-42c0-ac2a-643c51fd44f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:25.207977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4772a-c6b0-42c0-ac2a-643c51fd44f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:25.208265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4772a-c6b0-42c0-ac2a-643c51fd44f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:25.208570 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4772a-c6b0-42c0-ac2a-643c51fd44f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:25.219942 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4772a-c6b0-42c0-ac2a-643c51fd44f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:25.221435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4772a-c6b0-42c0-ac2a-643c51fd44f7 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:25.231579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 979/1985] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:26.356920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b912520-c15a-4647-9d02-15918ae7e60a None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:26.357422 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b912520-c15a-4647-9d02-15918ae7e60a None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:26.357575 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b912520-c15a-4647-9d02-15918ae7e60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:26.398638 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-4b912520-c15a-4647-9d02-15918ae7e60a None None] MFA Rules not processed for user `9e54c22586b74506b3246eab8de57807`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:26.400924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-4b912520-c15a-4647-9d02-15918ae7e60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:26.481739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1007/1986] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:26.485660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4772a-c6b0-42c0-ac2a-643c51fd44f7 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:26.496936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4772a-c6b0-42c0-ac2a-643c51fd44f7 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:26.502920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bd502-a49c-48cc-a139-58551790fa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:26.503609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd502-a49c-48cc-a139-58551790fa9e admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:26.504021 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd502-a49c-48cc-a139-58551790fa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:26.504307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd502-a49c-48cc-a139-58551790fa9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:26.505081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934bd502-a49c-48cc-a139-58551790fa9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:26.507106 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934bd502-a49c-48cc-a139-58551790fa9e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:26.528749 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:26.541204 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 980/1987] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:26 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:26.542053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b912520-c15a-4647-9d02-15918ae7e60a None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:26.555046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b912520-c15a-4647-9d02-15918ae7e60a None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:26.611276 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86807432-d095-44fb-9124-b13f9eb82941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:26.611993 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86807432-d095-44fb-9124-b13f9eb82941 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:26.612279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86807432-d095-44fb-9124-b13f9eb82941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:26.612559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86807432-d095-44fb-9124-b13f9eb82941 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:26.613311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86807432-d095-44fb-9124-b13f9eb82941 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:26.615255 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86807432-d095-44fb-9124-b13f9eb82941 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:26.636181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1008/1988] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:26 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:31:26.640353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bd502-a49c-48cc-a139-58551790fa9e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:26.651578 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bd502-a49c-48cc-a139-58551790fa9e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:26.656063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be444ca9-b06f-4f92-bf69-a3cbe67e532c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:26.656720 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be444ca9-b06f-4f92-bf69-a3cbe67e532c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:26.657001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be444ca9-b06f-4f92-bf69-a3cbe67e532c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:26.657347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be444ca9-b06f-4f92-bf69-a3cbe67e532c admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:26.658075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be444ca9-b06f-4f92-bf69-a3cbe67e532c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:26.659658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be444ca9-b06f-4f92-bf69-a3cbe67e532c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:26.670261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 981/1989] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:26.674229 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86807432-d095-44fb-9124-b13f9eb82941 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:26.684233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86807432-d095-44fb-9124-b13f9eb82941 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:26.689393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb339c-c76a-4448-a00a-a1ac07a47407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:26.690274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb339c-c76a-4448-a00a-a1ac07a47407 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:26.690557 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb339c-c76a-4448-a00a-a1ac07a47407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:26.690834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb339c-c76a-4448-a00a-a1ac07a47407 admin admin] PATH_INFO: `/v3/projects/5f79fcbe275b42a1b8cc61f890657afc/users/99f04eb288ed4f7f86f19dceca276d0c/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:26.705574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb339c-c76a-4448-a00a-a1ac07a47407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f79fcbe275b42a1b8cc61f890657afc, user_id=99f04eb288ed4f7f86f19dceca276d0c, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:26.708217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb339c-c76a-4448-a00a-a1ac07a47407 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:26.721654 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1009/1990] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:26 2022] PUT /identity/v3/projects/5f79fcbe275b42a1b8cc61f890657afc/users/99f04eb288ed4f7f86f19dceca276d0c/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:31:26.725535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be444ca9-b06f-4f92-bf69-a3cbe67e532c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:26.736020 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be444ca9-b06f-4f92-bf69-a3cbe67e532c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:26.795108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e2aee3-46df-411a-a09e-daa8fc83ee2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:26.795809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2aee3-46df-411a-a09e-daa8fc83ee2c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:26.796099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2aee3-46df-411a-a09e-daa8fc83ee2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:26.796382 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2aee3-46df-411a-a09e-daa8fc83ee2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:26.797151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e2aee3-46df-411a-a09e-daa8fc83ee2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:26.798776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e2aee3-46df-411a-a09e-daa8fc83ee2c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:26.812859 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 982/1991] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:26.814112 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb339c-c76a-4448-a00a-a1ac07a47407 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:26.825702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb339c-c76a-4448-a00a-a1ac07a47407 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:26.830291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57ceb1-8b06-4631-bebf-24da776d6310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:26.831163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57ceb1-8b06-4631-bebf-24da776d6310 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:26.831447 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57ceb1-8b06-4631-bebf-24da776d6310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:26.831724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57ceb1-8b06-4631-bebf-24da776d6310 admin admin] PATH_INFO: `/v3/projects/5f79fcbe275b42a1b8cc61f890657afc/users/99f04eb288ed4f7f86f19dceca276d0c/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:26.833747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c57ceb1-8b06-4631-bebf-24da776d6310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f79fcbe275b42a1b8cc61f890657afc, user_id=99f04eb288ed4f7f86f19dceca276d0c, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:26.837496 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c57ceb1-8b06-4631-bebf-24da776d6310 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:26.847720 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1010/1992] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:26 2022] PUT /identity/v3/projects/5f79fcbe275b42a1b8cc61f890657afc/users/99f04eb288ed4f7f86f19dceca276d0c/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:27.467788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97f154-737c-46a5-b793-0be3183c674a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:27.468097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97f154-737c-46a5-b793-0be3183c674a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:27.469008 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97f154-737c-46a5-b793-0be3183c674a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:27.501865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6f97f154-737c-46a5-b793-0be3183c674a None None] MFA Rules not processed for user `99f04eb288ed4f7f86f19dceca276d0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:27.503380 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6f97f154-737c-46a5-b793-0be3183c674a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:27.580267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 983/1993] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:27.592464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57ceb1-8b06-4631-bebf-24da776d6310 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:27.595034 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5c57ceb1-8b06-4631-bebf-24da776d6310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:27.615893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:27.669717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57ceb1-8b06-4631-bebf-24da776d6310 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:27.674901 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1c46d-c3b2-43a4-b15f-bcb2537158ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:27.675651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1c46d-c3b2-43a4-b15f-bcb2537158ad service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:27.676011 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1c46d-c3b2-43a4-b15f-bcb2537158ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:27.676365 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1c46d-c3b2-43a4-b15f-bcb2537158ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:27.733820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1c46d-c3b2-43a4-b15f-bcb2537158ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:27.735273 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1c46d-c3b2-43a4-b15f-bcb2537158ad service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:27.744231 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1011/1994] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:33.302827 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97f154-737c-46a5-b793-0be3183c674a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:33.304280 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6f97f154-737c-46a5-b793-0be3183c674a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:33.324869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97f154-737c-46a5-b793-0be3183c674a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:33.329101 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276dbc8-c072-4af4-b460-ad68f0c33af7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:33.329824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276dbc8-c072-4af4-b460-ad68f0c33af7 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:33.330112 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276dbc8-c072-4af4-b460-ad68f0c33af7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:33.330412 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276dbc8-c072-4af4-b460-ad68f0c33af7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:33.341852 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276dbc8-c072-4af4-b460-ad68f0c33af7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:33.343283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276dbc8-c072-4af4-b460-ad68f0c33af7 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:33.354533 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 984/1995] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:35.575769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1c46d-c3b2-43a4-b15f-bcb2537158ad service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:35.577215 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e1e1c46d-c3b2-43a4-b15f-bcb2537158ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:35.646850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1c46d-c3b2-43a4-b15f-bcb2537158ad service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:35.652228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96712cff-3c61-4031-8aaa-0556bc906427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:35.653105 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96712cff-3c61-4031-8aaa-0556bc906427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:35.653418 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96712cff-3c61-4031-8aaa-0556bc906427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:35.653695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96712cff-3c61-4031-8aaa-0556bc906427 admin admin] PATH_INFO: `/v3/users/cd5e6483f72c4b068630a894a28ce421` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:35.654927 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96712cff-3c61-4031-8aaa-0556bc906427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd5e6483f72c4b068630a894a28ce421)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:35.656880 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96712cff-3c61-4031-8aaa-0556bc906427 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:35.703498 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-96712cff-3c61-4031-8aaa-0556bc906427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd5e6483f72c4b068630a894a28ce421'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:35.715238 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-96712cff-3c61-4031-8aaa-0556bc906427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd5e6483f72c4b068630a894a28ce421'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:35.715887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-96712cff-3c61-4031-8aaa-0556bc906427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd5e6483f72c4b068630a894a28ce421'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:35.723143 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-96712cff-3c61-4031-8aaa-0556bc906427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd5e6483f72c4b068630a894a28ce421'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:35.723757 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-96712cff-3c61-4031-8aaa-0556bc906427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd5e6483f72c4b068630a894a28ce421'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:35.724957 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:35.739419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1012/1996] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:35 2022] DELETE /identity/v3/users/cd5e6483f72c4b068630a894a28ce421 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:35.744660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fc3cb-04ab-4077-9aaa-699f7f7a93fa None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:35.745178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fc3cb-04ab-4077-9aaa-699f7f7a93fa None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:35.745295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fc3cb-04ab-4077-9aaa-699f7f7a93fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:35.763396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b35fc3cb-04ab-4077-9aaa-699f7f7a93fa None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:35.764599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b35fc3cb-04ab-4077-9aaa-699f7f7a93fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:35.813757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:35.827111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 985/1997] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:35.836111 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96712cff-3c61-4031-8aaa-0556bc906427 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:35.837569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-96712cff-3c61-4031-8aaa-0556bc906427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:35.897466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96712cff-3c61-4031-8aaa-0556bc906427 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:35.901718 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27969c0-fea5-4c48-bf93-c8b76339ca48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:35.902393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27969c0-fea5-4c48-bf93-c8b76339ca48 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:35.902668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27969c0-fea5-4c48-bf93-c8b76339ca48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:35.902946 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27969c0-fea5-4c48-bf93-c8b76339ca48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:35.912658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27969c0-fea5-4c48-bf93-c8b76339ca48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:35.915005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27969c0-fea5-4c48-bf93-c8b76339ca48 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:35.925005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1013/1998] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:36.244390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fc3cb-04ab-4077-9aaa-699f7f7a93fa None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:36.246003 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b35fc3cb-04ab-4077-9aaa-699f7f7a93fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:36.274105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fc3cb-04ab-4077-9aaa-699f7f7a93fa None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:36.280904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347016c-d64d-4d5d-be06-ec7deed1cadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:36.282155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347016c-d64d-4d5d-be06-ec7deed1cadf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:36.282342 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347016c-d64d-4d5d-be06-ec7deed1cadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:36.295481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347016c-d64d-4d5d-be06-ec7deed1cadf admin admin] PATH_INFO: `/v3/projects/b01436b67f11490698d23fe930a4a3a0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:36.297909 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347016c-d64d-4d5d-be06-ec7deed1cadf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b01436b67f11490698d23fe930a4a3a0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:36.300261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347016c-d64d-4d5d-be06-ec7deed1cadf admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:36.370177 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1347016c-d64d-4d5d-be06-ec7deed1cadf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b01436b67f11490698d23fe930a4a3a0', 'request_id': 'req-1347016c-d64d-4d5d-be06-ec7deed1cadf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:36.378533 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1347016c-d64d-4d5d-be06-ec7deed1cadf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b01436b67f11490698d23fe930a4a3a0', 'request_id': 'req-1347016c-d64d-4d5d-be06-ec7deed1cadf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:36.386097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1347016c-d64d-4d5d-be06-ec7deed1cadf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b01436b67f11490698d23fe930a4a3a0', 'request_id': 'req-1347016c-d64d-4d5d-be06-ec7deed1cadf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:36.403812 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1347016c-d64d-4d5d-be06-ec7deed1cadf admin admin] The token cache is being invalidate because project b01436b67f11490698d23fe930a4a3a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:36.404180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1347016c-d64d-4d5d-be06-ec7deed1cadf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:36.405230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1347016c-d64d-4d5d-be06-ec7deed1cadf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:36.407592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 986/1999] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:36 2022] DELETE /identity/v3/projects/b01436b67f11490698d23fe930a4a3a0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:36.413323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27969c0-fea5-4c48-bf93-c8b76339ca48 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:36.424842 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27969c0-fea5-4c48-bf93-c8b76339ca48 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:36.493710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99bff1-f97b-4e90-8b2d-1039c2055d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:36.494764 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99bff1-f97b-4e90-8b2d-1039c2055d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:36.495083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99bff1-f97b-4e90-8b2d-1039c2055d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:36.495365 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99bff1-f97b-4e90-8b2d-1039c2055d22 admin admin] PATH_INFO: `/v3/users/62858da8658144eb92b6691cee0b3eda` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:36.496646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99bff1-f97b-4e90-8b2d-1039c2055d22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62858da8658144eb92b6691cee0b3eda)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:36.498647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99bff1-f97b-4e90-8b2d-1039c2055d22 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:36.545985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5e99bff1-f97b-4e90-8b2d-1039c2055d22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62858da8658144eb92b6691cee0b3eda'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:36.559201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5e99bff1-f97b-4e90-8b2d-1039c2055d22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62858da8658144eb92b6691cee0b3eda'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:36.559932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5e99bff1-f97b-4e90-8b2d-1039c2055d22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62858da8658144eb92b6691cee0b3eda'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:36.566660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5e99bff1-f97b-4e90-8b2d-1039c2055d22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62858da8658144eb92b6691cee0b3eda'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:36.567329 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5e99bff1-f97b-4e90-8b2d-1039c2055d22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62858da8658144eb92b6691cee0b3eda'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:36.581156 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1014/2000] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:36 2022] DELETE /identity/v3/users/62858da8658144eb92b6691cee0b3eda => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:31:36.852953 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347016c-d64d-4d5d-be06-ec7deed1cadf admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:36.854583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1347016c-d64d-4d5d-be06-ec7deed1cadf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:36.868091 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347016c-d64d-4d5d-be06-ec7deed1cadf admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:36.927385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef0f93-01e6-4342-8b36-ee3713092b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:36.928259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef0f93-01e6-4342-8b36-ee3713092b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:36.928595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef0f93-01e6-4342-8b36-ee3713092b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:36.928930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef0f93-01e6-4342-8b36-ee3713092b8b admin admin] PATH_INFO: `/v3/projects/b8c7289737cd4b3783ef6aa14394622b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:36.930147 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef0f93-01e6-4342-8b36-ee3713092b8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8c7289737cd4b3783ef6aa14394622b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:36.933635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef0f93-01e6-4342-8b36-ee3713092b8b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:36.981399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e1ef0f93-01e6-4342-8b36-ee3713092b8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8c7289737cd4b3783ef6aa14394622b', 'request_id': 'req-e1ef0f93-01e6-4342-8b36-ee3713092b8b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:36.988409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e1ef0f93-01e6-4342-8b36-ee3713092b8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8c7289737cd4b3783ef6aa14394622b', 'request_id': 'req-e1ef0f93-01e6-4342-8b36-ee3713092b8b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:36.993900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e1ef0f93-01e6-4342-8b36-ee3713092b8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8c7289737cd4b3783ef6aa14394622b', 'request_id': 'req-e1ef0f93-01e6-4342-8b36-ee3713092b8b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:37.004847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e1ef0f93-01e6-4342-8b36-ee3713092b8b admin admin] The token cache is being invalidate because project b8c7289737cd4b3783ef6aa14394622b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:37.005159 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e1ef0f93-01e6-4342-8b36-ee3713092b8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:37.005893 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e1ef0f93-01e6-4342-8b36-ee3713092b8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:37.007485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 987/2001] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:36 2022] DELETE /identity/v3/projects/b8c7289737cd4b3783ef6aa14394622b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:37.012836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3db7f9-3f48-4d67-9f8c-849ba05517b0 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:37.013122 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3db7f9-3f48-4d67-9f8c-849ba05517b0 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:37.013427 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3db7f9-3f48-4d67-9f8c-849ba05517b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:37.033467 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ad3db7f9-3f48-4d67-9f8c-849ba05517b0 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:37.034660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ad3db7f9-3f48-4d67-9f8c-849ba05517b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:37.040548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:37.095021 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1015/2002] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:37.103439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949ed99-cdb4-4220-9d2f-afd399918fcd None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:37.103745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949ed99-cdb4-4220-9d2f-afd399918fcd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:37.104030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949ed99-cdb4-4220-9d2f-afd399918fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:37.123833 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8949ed99-cdb4-4220-9d2f-afd399918fcd None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:37.125109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8949ed99-cdb4-4220-9d2f-afd399918fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:37.134239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 988/2003] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:37.138598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3db7f9-3f48-4d67-9f8c-849ba05517b0 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:37.148972 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3db7f9-3f48-4d67-9f8c-849ba05517b0 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:37.153782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736cdbb-451e-41fb-ac60-bf99bb724e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:37.154433 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736cdbb-451e-41fb-ac60-bf99bb724e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:37.154717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736cdbb-451e-41fb-ac60-bf99bb724e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:37.154993 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736cdbb-451e-41fb-ac60-bf99bb724e3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:37.155722 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736cdbb-451e-41fb-ac60-bf99bb724e3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:31:37.156030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736cdbb-451e-41fb-ac60-bf99bb724e3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:37.157675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736cdbb-451e-41fb-ac60-bf99bb724e3a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:37.169310 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1016/2004] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:31:37.173491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949ed99-cdb4-4220-9d2f-afd399918fcd None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:37.177935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:37.184569 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949ed99-cdb4-4220-9d2f-afd399918fcd None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:37.190509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c41139-5214-44e1-b76e-624ffe25a49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:37.191202 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c41139-5214-44e1-b76e-624ffe25a49a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:37.191490 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c41139-5214-44e1-b76e-624ffe25a49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:37.191771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c41139-5214-44e1-b76e-624ffe25a49a admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:37.192467 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c41139-5214-44e1-b76e-624ffe25a49a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:37.194915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c41139-5214-44e1-b76e-624ffe25a49a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:37.221783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 989/2005] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:37 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:37.226176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736cdbb-451e-41fb-ac60-bf99bb724e3a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:37.237938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736cdbb-451e-41fb-ac60-bf99bb724e3a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:37.313453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f5aed-fcf1-4643-a894-2c13ea0129a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:37.314837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f5aed-fcf1-4643-a894-2c13ea0129a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:37.315198 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f5aed-fcf1-4643-a894-2c13ea0129a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:37.315481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f5aed-fcf1-4643-a894-2c13ea0129a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:37.316192 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f5aed-fcf1-4643-a894-2c13ea0129a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:37.318243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f5aed-fcf1-4643-a894-2c13ea0129a9 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:37.339564 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1017/2006] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:37 2022] POST /identity/v3/users => generated 282 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:37.343895 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c41139-5214-44e1-b76e-624ffe25a49a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:37.355686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c41139-5214-44e1-b76e-624ffe25a49a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:37.361006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a778a3-0e8a-4fab-9362-ddecdaff55ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:37.361781 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a778a3-0e8a-4fab-9362-ddecdaff55ff admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:37.362168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a778a3-0e8a-4fab-9362-ddecdaff55ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:37.362451 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a778a3-0e8a-4fab-9362-ddecdaff55ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:37.363185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a778a3-0e8a-4fab-9362-ddecdaff55ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:37.364793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a778a3-0e8a-4fab-9362-ddecdaff55ff admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:37.376577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 990/2007] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:37.380662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f5aed-fcf1-4643-a894-2c13ea0129a9 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:37.392950 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f5aed-fcf1-4643-a894-2c13ea0129a9 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:37.400027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258624b-bc20-42d1-832b-69b1b7d6539e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:37.400711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258624b-bc20-42d1-832b-69b1b7d6539e admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:37.400996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258624b-bc20-42d1-832b-69b1b7d6539e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:37.401364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258624b-bc20-42d1-832b-69b1b7d6539e admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:37.402110 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258624b-bc20-42d1-832b-69b1b7d6539e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:37.403722 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258624b-bc20-42d1-832b-69b1b7d6539e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:37.415159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1018/2008] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:37.419323 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a778a3-0e8a-4fab-9362-ddecdaff55ff admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:37.430705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a778a3-0e8a-4fab-9362-ddecdaff55ff admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:37.435631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85533f45-e913-44a7-b352-885940a464bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:37.436700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85533f45-e913-44a7-b352-885940a464bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:37.437081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85533f45-e913-44a7-b352-885940a464bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:37.437492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85533f45-e913-44a7-b352-885940a464bc admin admin] PATH_INFO: `/v3/projects/c8ef1272050c4b3aa5dc7d7396a01073/users/d70396f0531441dbb1ae6eb7b31867ed/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:37.465053 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85533f45-e913-44a7-b352-885940a464bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ef1272050c4b3aa5dc7d7396a01073, user_id=d70396f0531441dbb1ae6eb7b31867ed, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:37.467804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85533f45-e913-44a7-b352-885940a464bc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:37.489158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 991/2009] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:37 2022] PUT /identity/v3/projects/c8ef1272050c4b3aa5dc7d7396a01073/users/d70396f0531441dbb1ae6eb7b31867ed/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:37.501663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c562b3-6fb6-49bf-9ada-7a8eca1b03a9 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:37.501998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c562b3-6fb6-49bf-9ada-7a8eca1b03a9 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:37.502290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c562b3-6fb6-49bf-9ada-7a8eca1b03a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:37.545653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-12c562b3-6fb6-49bf-9ada-7a8eca1b03a9 None None] MFA Rules not processed for user `d70396f0531441dbb1ae6eb7b31867ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:37.547036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-12c562b3-6fb6-49bf-9ada-7a8eca1b03a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:37.629856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85533f45-e913-44a7-b352-885940a464bc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:37.630880 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1019/2010] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:37 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:37.736924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85533f45-e913-44a7-b352-885940a464bc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:37.753377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0dffd-39d8-44a6-82f0-6a18d37ba786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:37.753377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0dffd-39d8-44a6-82f0-6a18d37ba786 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:37.753377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0dffd-39d8-44a6-82f0-6a18d37ba786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:37.753377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0dffd-39d8-44a6-82f0-6a18d37ba786 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:37.753909 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0dffd-39d8-44a6-82f0-6a18d37ba786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:37.753909 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0dffd-39d8-44a6-82f0-6a18d37ba786 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:37.791532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 992/2011] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:37 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:31:37.797021 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c562b3-6fb6-49bf-9ada-7a8eca1b03a9 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:37.812922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c562b3-6fb6-49bf-9ada-7a8eca1b03a9 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:37.816550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:37.872118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95a123-83d4-4a1b-a50b-0bf519621ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:37.872802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95a123-83d4-4a1b-a50b-0bf519621ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:37.873153 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95a123-83d4-4a1b-a50b-0bf519621ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:37.873450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95a123-83d4-4a1b-a50b-0bf519621ac0 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:37.874155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95a123-83d4-4a1b-a50b-0bf519621ac0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:37.876089 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95a123-83d4-4a1b-a50b-0bf519621ac0 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:37.898311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1020/2012] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:37 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:37.902627 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0dffd-39d8-44a6-82f0-6a18d37ba786 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:37.913124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0dffd-39d8-44a6-82f0-6a18d37ba786 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:37.917935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f269a6-0433-43b1-bcb0-54865534791e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:37.918617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f269a6-0433-43b1-bcb0-54865534791e admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:37.918899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f269a6-0433-43b1-bcb0-54865534791e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:37.919175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f269a6-0433-43b1-bcb0-54865534791e admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:37.919896 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f269a6-0433-43b1-bcb0-54865534791e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:37.921523 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f269a6-0433-43b1-bcb0-54865534791e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:37.932774 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 993/2013] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:37.936750 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95a123-83d4-4a1b-a50b-0bf519621ac0 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:37.947381 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95a123-83d4-4a1b-a50b-0bf519621ac0 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:37.952276 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4d23a-7f0c-4054-b2a7-f2b5f234b664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:37.953342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4d23a-7f0c-4054-b2a7-f2b5f234b664 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:37.953752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4d23a-7f0c-4054-b2a7-f2b5f234b664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:37.954043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4d23a-7f0c-4054-b2a7-f2b5f234b664 admin admin] PATH_INFO: `/v3/projects/60a4d48d2f9043e48a2bfde9ed8317ab/users/9ab63c6cb31c494383b7101cfcd5da27/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:37.970485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c4d23a-7f0c-4054-b2a7-f2b5f234b664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a4d48d2f9043e48a2bfde9ed8317ab, user_id=9ab63c6cb31c494383b7101cfcd5da27, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:37.973332 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c4d23a-7f0c-4054-b2a7-f2b5f234b664 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:37.987582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1021/2014] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:37 2022] PUT /identity/v3/projects/60a4d48d2f9043e48a2bfde9ed8317ab/users/9ab63c6cb31c494383b7101cfcd5da27/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:37.991522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f269a6-0433-43b1-bcb0-54865534791e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:38.001820 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f269a6-0433-43b1-bcb0-54865534791e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:38.064556 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72adeaf3-172a-4013-a74c-dae12a13835f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:38.065271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adeaf3-172a-4013-a74c-dae12a13835f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:38.065543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adeaf3-172a-4013-a74c-dae12a13835f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:38.065818 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adeaf3-172a-4013-a74c-dae12a13835f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:38.066553 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72adeaf3-172a-4013-a74c-dae12a13835f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:38.068188 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72adeaf3-172a-4013-a74c-dae12a13835f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:38.078773 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 994/2015] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:38.082847 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4d23a-7f0c-4054-b2a7-f2b5f234b664 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:38.094006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4d23a-7f0c-4054-b2a7-f2b5f234b664 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:38.098815 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548afa8-331b-4a91-8510-3c637f97dcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:38.099705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548afa8-331b-4a91-8510-3c637f97dcdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:38.099990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548afa8-331b-4a91-8510-3c637f97dcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:38.100377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548afa8-331b-4a91-8510-3c637f97dcdb admin admin] PATH_INFO: `/v3/projects/60a4d48d2f9043e48a2bfde9ed8317ab/users/9ab63c6cb31c494383b7101cfcd5da27/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:38.102645 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548afa8-331b-4a91-8510-3c637f97dcdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a4d48d2f9043e48a2bfde9ed8317ab, user_id=9ab63c6cb31c494383b7101cfcd5da27, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:38.105733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548afa8-331b-4a91-8510-3c637f97dcdb admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:38.116577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1022/2016] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:38 2022] PUT /identity/v3/projects/60a4d48d2f9043e48a2bfde9ed8317ab/users/9ab63c6cb31c494383b7101cfcd5da27/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:38.127710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00555e-fc95-4e3a-bed7-94e830fa9114 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:38.128004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00555e-fc95-4e3a-bed7-94e830fa9114 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:38.128285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00555e-fc95-4e3a-bed7-94e830fa9114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:38.156332 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ec00555e-fc95-4e3a-bed7-94e830fa9114 None None] MFA Rules not processed for user `9ab63c6cb31c494383b7101cfcd5da27`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:38.157584 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ec00555e-fc95-4e3a-bed7-94e830fa9114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:38.161804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:38.230936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 995/2017] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:38 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:38.243920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548afa8-331b-4a91-8510-3c637f97dcdb admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:38.246199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5548afa8-331b-4a91-8510-3c637f97dcdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:38.306696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548afa8-331b-4a91-8510-3c637f97dcdb admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:38.310654 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f506d-db69-47e9-a60b-dd6c196bd4a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:38.311303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f506d-db69-47e9-a60b-dd6c196bd4a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:38.311590 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f506d-db69-47e9-a60b-dd6c196bd4a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:38.311873 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f506d-db69-47e9-a60b-dd6c196bd4a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:38.321216 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f506d-db69-47e9-a60b-dd6c196bd4a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:38.322662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f506d-db69-47e9-a60b-dd6c196bd4a5 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:38.332617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1023/2018] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:38 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:38.779205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00555e-fc95-4e3a-bed7-94e830fa9114 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:38.854090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00555e-fc95-4e3a-bed7-94e830fa9114 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:38.858665 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6275a2-db6a-4ee3-afc1-aecbda1a4f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:38.859596 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6275a2-db6a-4ee3-afc1-aecbda1a4f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:38.859885 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6275a2-db6a-4ee3-afc1-aecbda1a4f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:38.860167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6275a2-db6a-4ee3-afc1-aecbda1a4f9d admin admin] PATH_INFO: `/v3/users/d70396f0531441dbb1ae6eb7b31867ed` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:38.861392 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6275a2-db6a-4ee3-afc1-aecbda1a4f9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d70396f0531441dbb1ae6eb7b31867ed)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:38.863348 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6275a2-db6a-4ee3-afc1-aecbda1a4f9d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:38.913926 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0b6275a2-db6a-4ee3-afc1-aecbda1a4f9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd70396f0531441dbb1ae6eb7b31867ed'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:38.926326 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0b6275a2-db6a-4ee3-afc1-aecbda1a4f9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd70396f0531441dbb1ae6eb7b31867ed'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:38.927023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0b6275a2-db6a-4ee3-afc1-aecbda1a4f9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd70396f0531441dbb1ae6eb7b31867ed'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:38.927635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0b6275a2-db6a-4ee3-afc1-aecbda1a4f9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd70396f0531441dbb1ae6eb7b31867ed'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:38.935131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0b6275a2-db6a-4ee3-afc1-aecbda1a4f9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd70396f0531441dbb1ae6eb7b31867ed'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:38.950308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 996/2019] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:38 2022] DELETE /identity/v3/users/d70396f0531441dbb1ae6eb7b31867ed => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:38.954888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f29df-a9b0-494d-95d6-a9d803582052 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:38.955779 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f29df-a9b0-494d-95d6-a9d803582052 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:38.956063 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f29df-a9b0-494d-95d6-a9d803582052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:38.974245 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-0e2f29df-a9b0-494d-95d6-a9d803582052 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:38.975560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0e2f29df-a9b0-494d-95d6-a9d803582052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:39.036971 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1024/2020] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:31:39.046634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6275a2-db6a-4ee3-afc1-aecbda1a4f9d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:39.048057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0b6275a2-db6a-4ee3-afc1-aecbda1a4f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:39.072500 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:39.112932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6275a2-db6a-4ee3-afc1-aecbda1a4f9d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:39.117780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47a872-d90e-4a8e-80d8-72df2e8352fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:39.118441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47a872-d90e-4a8e-80d8-72df2e8352fe service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:39.118795 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47a872-d90e-4a8e-80d8-72df2e8352fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:39.119076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47a872-d90e-4a8e-80d8-72df2e8352fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:39.128826 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47a872-d90e-4a8e-80d8-72df2e8352fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:39.130316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47a872-d90e-4a8e-80d8-72df2e8352fe service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:39.142091 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 997/2021] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:39.636361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f29df-a9b0-494d-95d6-a9d803582052 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:39.638647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0e2f29df-a9b0-494d-95d6-a9d803582052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:39.652837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:39.666839 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f29df-a9b0-494d-95d6-a9d803582052 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:39.671986 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e7027-0f4f-4b29-979d-34c60592c1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:39.673036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e7027-0f4f-4b29-979d-34c60592c1cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:39.673528 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e7027-0f4f-4b29-979d-34c60592c1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:39.673952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e7027-0f4f-4b29-979d-34c60592c1cc admin admin] PATH_INFO: `/v3/projects/c8ef1272050c4b3aa5dc7d7396a01073` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:39.675318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e7027-0f4f-4b29-979d-34c60592c1cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ef1272050c4b3aa5dc7d7396a01073)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:39.677527 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e7027-0f4f-4b29-979d-34c60592c1cc admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:39.740483 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c48e7027-0f4f-4b29-979d-34c60592c1cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ef1272050c4b3aa5dc7d7396a01073', 'request_id': 'req-c48e7027-0f4f-4b29-979d-34c60592c1cc'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:39.747302 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c48e7027-0f4f-4b29-979d-34c60592c1cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ef1272050c4b3aa5dc7d7396a01073', 'request_id': 'req-c48e7027-0f4f-4b29-979d-34c60592c1cc'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:39.754349 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c48e7027-0f4f-4b29-979d-34c60592c1cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ef1272050c4b3aa5dc7d7396a01073', 'request_id': 'req-c48e7027-0f4f-4b29-979d-34c60592c1cc'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:39.769222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c48e7027-0f4f-4b29-979d-34c60592c1cc admin admin] The token cache is being invalidate because project c8ef1272050c4b3aa5dc7d7396a01073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:39.769522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c48e7027-0f4f-4b29-979d-34c60592c1cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:39.770184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c48e7027-0f4f-4b29-979d-34c60592c1cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:39.772589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1025/2022] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:39 2022] DELETE /identity/v3/projects/c8ef1272050c4b3aa5dc7d7396a01073 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:39.777613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47a872-d90e-4a8e-80d8-72df2e8352fe service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:39.789348 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47a872-d90e-4a8e-80d8-72df2e8352fe service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:39.862026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef25a4b-cb20-429c-a4d9-d8f96c1afc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:39.863014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef25a4b-cb20-429c-a4d9-d8f96c1afc17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:39.863283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef25a4b-cb20-429c-a4d9-d8f96c1afc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:39.864510 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef25a4b-cb20-429c-a4d9-d8f96c1afc17 admin admin] PATH_INFO: `/v3/users/9ab63c6cb31c494383b7101cfcd5da27` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:39.865313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef25a4b-cb20-429c-a4d9-d8f96c1afc17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab63c6cb31c494383b7101cfcd5da27)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:39.868570 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef25a4b-cb20-429c-a4d9-d8f96c1afc17 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:39.931883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fef25a4b-cb20-429c-a4d9-d8f96c1afc17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab63c6cb31c494383b7101cfcd5da27'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:39.949928 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fef25a4b-cb20-429c-a4d9-d8f96c1afc17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab63c6cb31c494383b7101cfcd5da27'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:39.952462 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fef25a4b-cb20-429c-a4d9-d8f96c1afc17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab63c6cb31c494383b7101cfcd5da27'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:39.953133 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fef25a4b-cb20-429c-a4d9-d8f96c1afc17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab63c6cb31c494383b7101cfcd5da27'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:39.962286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fef25a4b-cb20-429c-a4d9-d8f96c1afc17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab63c6cb31c494383b7101cfcd5da27'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:39.982286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 998/2023] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:39 2022] DELETE /identity/v3/users/9ab63c6cb31c494383b7101cfcd5da27 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:40.313862 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e7027-0f4f-4b29-979d-34c60592c1cc admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:40.315454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c48e7027-0f4f-4b29-979d-34c60592c1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:40.333316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e7027-0f4f-4b29-979d-34c60592c1cc admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:40.419489 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0bff4-600a-4161-8af8-8307db5eba69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:40.420407 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0bff4-600a-4161-8af8-8307db5eba69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:40.420687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0bff4-600a-4161-8af8-8307db5eba69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:40.420961 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0bff4-600a-4161-8af8-8307db5eba69 admin admin] PATH_INFO: `/v3/projects/60a4d48d2f9043e48a2bfde9ed8317ab` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:40.422297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0bff4-600a-4161-8af8-8307db5eba69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60a4d48d2f9043e48a2bfde9ed8317ab)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:40.424358 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0bff4-600a-4161-8af8-8307db5eba69 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:40.502310 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-41b0bff4-600a-4161-8af8-8307db5eba69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60a4d48d2f9043e48a2bfde9ed8317ab', 'request_id': 'req-41b0bff4-600a-4161-8af8-8307db5eba69'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:40.508605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-41b0bff4-600a-4161-8af8-8307db5eba69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60a4d48d2f9043e48a2bfde9ed8317ab', 'request_id': 'req-41b0bff4-600a-4161-8af8-8307db5eba69'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:40.514920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-41b0bff4-600a-4161-8af8-8307db5eba69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60a4d48d2f9043e48a2bfde9ed8317ab', 'request_id': 'req-41b0bff4-600a-4161-8af8-8307db5eba69'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:40.529840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-41b0bff4-600a-4161-8af8-8307db5eba69 admin admin] The token cache is being invalidate because project 60a4d48d2f9043e48a2bfde9ed8317ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:40.530139 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-41b0bff4-600a-4161-8af8-8307db5eba69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:40.530781 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-41b0bff4-600a-4161-8af8-8307db5eba69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:40.532354 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1026/2024] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:40 2022] DELETE /identity/v3/projects/60a4d48d2f9043e48a2bfde9ed8317ab => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:40.538553 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6275080-69c9-43db-a081-5215f11d48d1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:40.538908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6275080-69c9-43db-a081-5215f11d48d1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:40.539456 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6275080-69c9-43db-a081-5215f11d48d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:40.574105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c6275080-69c9-43db-a081-5215f11d48d1 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:40.575620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c6275080-69c9-43db-a081-5215f11d48d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:40.641153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:40.678266 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 999/2025] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:40.690178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b918f0b-5c07-47a8-b90e-051f1e2f6835 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:40.690178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b918f0b-5c07-47a8-b90e-051f1e2f6835 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:40.690178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b918f0b-5c07-47a8-b90e-051f1e2f6835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:40.719288 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-3b918f0b-5c07-47a8-b90e-051f1e2f6835 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:40.720847 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3b918f0b-5c07-47a8-b90e-051f1e2f6835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:40.738173 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1027/2026] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:40.739420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6275080-69c9-43db-a081-5215f11d48d1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:40.753561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6275080-69c9-43db-a081-5215f11d48d1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:40.759816 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710a177-7488-46be-ba11-6e16339ff863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:40.760497 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710a177-7488-46be-ba11-6e16339ff863 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:40.760778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710a177-7488-46be-ba11-6e16339ff863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:40.761092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710a177-7488-46be-ba11-6e16339ff863 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:40.761908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710a177-7488-46be-ba11-6e16339ff863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:31:40.762219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710a177-7488-46be-ba11-6e16339ff863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:40.767517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710a177-7488-46be-ba11-6e16339ff863 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:40.783262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1000/2027] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:31:40.787207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b918f0b-5c07-47a8-b90e-051f1e2f6835 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:40.807819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b918f0b-5c07-47a8-b90e-051f1e2f6835 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:40.837539 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bedff95-2026-4a32-925e-3ba7f5a4dc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:40.839943 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedff95-2026-4a32-925e-3ba7f5a4dc7f admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:40.840236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedff95-2026-4a32-925e-3ba7f5a4dc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:40.840518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedff95-2026-4a32-925e-3ba7f5a4dc7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:40.841260 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bedff95-2026-4a32-925e-3ba7f5a4dc7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:40.849373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bedff95-2026-4a32-925e-3ba7f5a4dc7f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:40.867148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:40.900299 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1028/2028] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:40 2022] POST /identity/v3/projects => generated 369 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:40.903070 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710a177-7488-46be-ba11-6e16339ff863 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:40.919147 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710a177-7488-46be-ba11-6e16339ff863 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:41.003845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5d723-5c44-4a63-8709-97fb70504508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:41.004611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5d723-5c44-4a63-8709-97fb70504508 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:41.004819 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5d723-5c44-4a63-8709-97fb70504508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:41.005174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5d723-5c44-4a63-8709-97fb70504508 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:41.006176 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f5d723-5c44-4a63-8709-97fb70504508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:41.009771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f5d723-5c44-4a63-8709-97fb70504508 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:41.040763 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1001/2029] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:40 2022] POST /identity/v3/users => generated 285 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:41.044863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bedff95-2026-4a32-925e-3ba7f5a4dc7f admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:41.060746 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bedff95-2026-4a32-925e-3ba7f5a4dc7f admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:41.072652 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ac0be-8874-4cd5-a618-11992cbcc8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:41.073632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ac0be-8874-4cd5-a618-11992cbcc8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:41.073965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ac0be-8874-4cd5-a618-11992cbcc8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:41.074248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ac0be-8874-4cd5-a618-11992cbcc8f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:41.074992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ac0be-8874-4cd5-a618-11992cbcc8f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:41.076660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ac0be-8874-4cd5-a618-11992cbcc8f0 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:41.107194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1029/2030] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:41.111133 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5d723-5c44-4a63-8709-97fb70504508 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:41.124262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5d723-5c44-4a63-8709-97fb70504508 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:41.131064 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c46f2e-041b-4348-924b-f973bf7699ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:41.132821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c46f2e-041b-4348-924b-f973bf7699ca admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:41.132821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c46f2e-041b-4348-924b-f973bf7699ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:41.132821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c46f2e-041b-4348-924b-f973bf7699ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:41.133615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c46f2e-041b-4348-924b-f973bf7699ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:41.135373 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c46f2e-041b-4348-924b-f973bf7699ca admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:41.148655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1002/2031] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:31:41.153243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ac0be-8874-4cd5-a618-11992cbcc8f0 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:41.165168 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ac0be-8874-4cd5-a618-11992cbcc8f0 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:41.169983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f263f34-7602-42b7-bb0f-3b90503ef80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:41.170878 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f263f34-7602-42b7-bb0f-3b90503ef80b admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:41.171161 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f263f34-7602-42b7-bb0f-3b90503ef80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:41.171438 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f263f34-7602-42b7-bb0f-3b90503ef80b admin admin] PATH_INFO: `/v3/projects/48aded3a02ee452bb8be3ae636d33238/users/793271de95d3456abf405051f7caf516/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:41.211731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f263f34-7602-42b7-bb0f-3b90503ef80b admin admin] RBAC: Authorizing `identity:create_grant(project_id=48aded3a02ee452bb8be3ae636d33238, user_id=793271de95d3456abf405051f7caf516, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:41.214933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f263f34-7602-42b7-bb0f-3b90503ef80b admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:41.236055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1030/2032] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:41 2022] PUT /identity/v3/projects/48aded3a02ee452bb8be3ae636d33238/users/793271de95d3456abf405051f7caf516/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:41.248731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b1003-0b86-4d65-8b58-1791624f231d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:41.249080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b1003-0b86-4d65-8b58-1791624f231d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:41.249402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b1003-0b86-4d65-8b58-1791624f231d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:41.290174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c09b1003-0b86-4d65-8b58-1791624f231d None None] MFA Rules not processed for user `793271de95d3456abf405051f7caf516`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:41.291690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c09b1003-0b86-4d65-8b58-1791624f231d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:41.373666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f263f34-7602-42b7-bb0f-3b90503ef80b admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:41.375662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2f263f34-7602-42b7-bb0f-3b90503ef80b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:41.377276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1003/2033] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:41.379340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b1003-0b86-4d65-8b58-1791624f231d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:41.426205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:41.451204 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f263f34-7602-42b7-bb0f-3b90503ef80b admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:41.456985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b1003-0b86-4d65-8b58-1791624f231d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:41.457112 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c47f4b-bdbc-43b6-a03f-8c6edbea4f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:41.457112 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c47f4b-bdbc-43b6-a03f-8c6edbea4f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:41.457370 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c47f4b-bdbc-43b6-a03f-8c6edbea4f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:41.457498 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c47f4b-bdbc-43b6-a03f-8c6edbea4f00 admin admin] PATH_INFO: `/v3/users/064d7bbcf3684ccbafc5f3d1cb561d9a` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:41.459175 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c47f4b-bdbc-43b6-a03f-8c6edbea4f00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=064d7bbcf3684ccbafc5f3d1cb561d9a)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:41.461417 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c47f4b-bdbc-43b6-a03f-8c6edbea4f00 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:41.462851 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30381ba-a3e9-49f4-a1b1-c316e0a9ca70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:41.463618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30381ba-a3e9-49f4-a1b1-c316e0a9ca70 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:41.463973 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30381ba-a3e9-49f4-a1b1-c316e0a9ca70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:41.464308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30381ba-a3e9-49f4-a1b1-c316e0a9ca70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:41.465083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30381ba-a3e9-49f4-a1b1-c316e0a9ca70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:41.467279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30381ba-a3e9-49f4-a1b1-c316e0a9ca70 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:41.497921 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1004/2034] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:41 2022] POST /identity/v3/projects => generated 367 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:41.501579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30381ba-a3e9-49f4-a1b1-c316e0a9ca70 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:41.519109 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-02c47f4b-bdbc-43b6-a03f-8c6edbea4f00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '064d7bbcf3684ccbafc5f3d1cb561d9a'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:41.533262 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-02c47f4b-bdbc-43b6-a03f-8c6edbea4f00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '064d7bbcf3684ccbafc5f3d1cb561d9a'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:41.534295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-02c47f4b-bdbc-43b6-a03f-8c6edbea4f00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '064d7bbcf3684ccbafc5f3d1cb561d9a'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:41.565124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-02c47f4b-bdbc-43b6-a03f-8c6edbea4f00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '064d7bbcf3684ccbafc5f3d1cb561d9a'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:41.566131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-02c47f4b-bdbc-43b6-a03f-8c6edbea4f00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '064d7bbcf3684ccbafc5f3d1cb561d9a'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:41.592472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30381ba-a3e9-49f4-a1b1-c316e0a9ca70 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:41.592853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1031/2035] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:41 2022] DELETE /identity/v3/users/064d7bbcf3684ccbafc5f3d1cb561d9a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:41.686939 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86153ca8-6948-4e1a-91f0-d8725e7b2dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:41.687761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86153ca8-6948-4e1a-91f0-d8725e7b2dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:41.688142 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86153ca8-6948-4e1a-91f0-d8725e7b2dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:41.688870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86153ca8-6948-4e1a-91f0-d8725e7b2dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:41.689348 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86153ca8-6948-4e1a-91f0-d8725e7b2dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:41.691752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86153ca8-6948-4e1a-91f0-d8725e7b2dd2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:41.719581 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1005/2036] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:41 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:41.720613 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c47f4b-bdbc-43b6-a03f-8c6edbea4f00 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:41.722115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-02c47f4b-bdbc-43b6-a03f-8c6edbea4f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:41.740081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c47f4b-bdbc-43b6-a03f-8c6edbea4f00 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:41.745574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cd018-b6c3-436c-8c21-c3680c512cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:41.746330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cd018-b6c3-436c-8c21-c3680c512cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:41.746712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cd018-b6c3-436c-8c21-c3680c512cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:41.747093 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cd018-b6c3-436c-8c21-c3680c512cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:41.747900 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3cd018-b6c3-436c-8c21-c3680c512cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:41.749630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3cd018-b6c3-436c-8c21-c3680c512cd1 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:41.761174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1032/2037] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:31:41.764970 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86153ca8-6948-4e1a-91f0-d8725e7b2dd2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:41.775710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86153ca8-6948-4e1a-91f0-d8725e7b2dd2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:41.780235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70cd7d-a002-4d02-8719-3e5a63b48afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:41.781176 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70cd7d-a002-4d02-8719-3e5a63b48afc admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:41.781463 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70cd7d-a002-4d02-8719-3e5a63b48afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:41.781737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70cd7d-a002-4d02-8719-3e5a63b48afc admin admin] PATH_INFO: `/v3/projects/1f42d8478cce4b3687d121e6b7277d27/users/7b0db8536cbf4042961d12f4064aab51/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:41.799543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70cd7d-a002-4d02-8719-3e5a63b48afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f42d8478cce4b3687d121e6b7277d27, user_id=7b0db8536cbf4042961d12f4064aab51, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:41.802268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70cd7d-a002-4d02-8719-3e5a63b48afc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:41.816038 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cd018-b6c3-436c-8c21-c3680c512cd1 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:41.817889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1006/2038] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:41 2022] PUT /identity/v3/projects/1f42d8478cce4b3687d121e6b7277d27/users/7b0db8536cbf4042961d12f4064aab51/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:41.818269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9c3cd018-b6c3-436c-8c21-c3680c512cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:41.821901 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70cd7d-a002-4d02-8719-3e5a63b48afc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:41.832010 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cd018-b6c3-436c-8c21-c3680c512cd1 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:41.849737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:41.868959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:41.893329 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a39830-16ed-4309-96cc-b8ccf4c30820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:41.893868 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70cd7d-a002-4d02-8719-3e5a63b48afc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:41.894201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a39830-16ed-4309-96cc-b8ccf4c30820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:41.894482 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a39830-16ed-4309-96cc-b8ccf4c30820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:41.894757 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a39830-16ed-4309-96cc-b8ccf4c30820 admin admin] PATH_INFO: `/v3/projects/2deb7182f86846e09269b7cba6fab758` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:41.895928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a39830-16ed-4309-96cc-b8ccf4c30820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2deb7182f86846e09269b7cba6fab758)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:41.898019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a39830-16ed-4309-96cc-b8ccf4c30820 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:41.899420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b56c93-afad-4e7b-8f43-b2ef4fdffe85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:41.900260 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b56c93-afad-4e7b-8f43-b2ef4fdffe85 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:41.900857 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b56c93-afad-4e7b-8f43-b2ef4fdffe85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:41.901155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b56c93-afad-4e7b-8f43-b2ef4fdffe85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:41.901889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b56c93-afad-4e7b-8f43-b2ef4fdffe85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:41.903483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b56c93-afad-4e7b-8f43-b2ef4fdffe85 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:41.916244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1007/2039] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:31:41.920658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b56c93-afad-4e7b-8f43-b2ef4fdffe85 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:41.931509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b56c93-afad-4e7b-8f43-b2ef4fdffe85 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:41.936626 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5a925-256e-46d9-81fc-fcfd1f0d7dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:41.937542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5a925-256e-46d9-81fc-fcfd1f0d7dd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:41.938185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5a925-256e-46d9-81fc-fcfd1f0d7dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:41.938347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5a925-256e-46d9-81fc-fcfd1f0d7dd3 admin admin] PATH_INFO: `/v3/projects/1f42d8478cce4b3687d121e6b7277d27/users/7b0db8536cbf4042961d12f4064aab51/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:41.940347 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5a925-256e-46d9-81fc-fcfd1f0d7dd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f42d8478cce4b3687d121e6b7277d27, user_id=7b0db8536cbf4042961d12f4064aab51, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:41.942948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5a925-256e-46d9-81fc-fcfd1f0d7dd3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:41.951543 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c6a39830-16ed-4309-96cc-b8ccf4c30820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2deb7182f86846e09269b7cba6fab758', 'request_id': 'req-c6a39830-16ed-4309-96cc-b8ccf4c30820'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:41.954980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1008/2040] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:41 2022] PUT /identity/v3/projects/1f42d8478cce4b3687d121e6b7277d27/users/7b0db8536cbf4042961d12f4064aab51/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:41.957376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c6a39830-16ed-4309-96cc-b8ccf4c30820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2deb7182f86846e09269b7cba6fab758', 'request_id': 'req-c6a39830-16ed-4309-96cc-b8ccf4c30820'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:41.964188 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c6a39830-16ed-4309-96cc-b8ccf4c30820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2deb7182f86846e09269b7cba6fab758', 'request_id': 'req-c6a39830-16ed-4309-96cc-b8ccf4c30820'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:41.967279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5cfb2-cade-416b-929e-c2181b257346 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:41.967576 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5cfb2-cade-416b-929e-c2181b257346 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:41.968171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5cfb2-cade-416b-929e-c2181b257346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:41.978701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c6a39830-16ed-4309-96cc-b8ccf4c30820 admin admin] The token cache is being invalidate because project 2deb7182f86846e09269b7cba6fab758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:41.979231 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c6a39830-16ed-4309-96cc-b8ccf4c30820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:41.979802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c6a39830-16ed-4309-96cc-b8ccf4c30820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:41.981631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1033/2041] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:41 2022] DELETE /identity/v3/projects/2deb7182f86846e09269b7cba6fab758 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:31:41.986271 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c3b83-2b55-4994-9f76-b818af50e5a2 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:41.986556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c3b83-2b55-4994-9f76-b818af50e5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:41.986868 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c3b83-2b55-4994-9f76-b818af50e5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:41.997434 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5cb5cfb2-cade-416b-929e-c2181b257346 None None] MFA Rules not processed for user `7b0db8536cbf4042961d12f4064aab51`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:41.998645 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5cb5cfb2-cade-416b-929e-c2181b257346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:42.006140 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-924c3b83-2b55-4994-9f76-b818af50e5a2 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:42.008003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-924c3b83-2b55-4994-9f76-b818af50e5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:42.071862 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1034/2042] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:31:42.078972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1009/2043] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:41 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:42.080144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b325db-5a57-43cf-8396-b9bbedf167f0 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:42.080442 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b325db-5a57-43cf-8396-b9bbedf167f0 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:42.080727 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b325db-5a57-43cf-8396-b9bbedf167f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:42.090550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5cfb2-cade-416b-929e-c2181b257346 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:42.091892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5cb5cfb2-cade-416b-929e-c2181b257346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:42.099449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-a4b325db-5a57-43cf-8396-b9bbedf167f0 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:42.101294 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a4b325db-5a57-43cf-8396-b9bbedf167f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:42.109328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1035/2044] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:31:42.113879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b325db-5a57-43cf-8396-b9bbedf167f0 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:42.124189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b325db-5a57-43cf-8396-b9bbedf167f0 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:42.128735 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e1c6c-2690-43c3-a4ad-3b7af36b0608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:42.129402 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e1c6c-2690-43c3-a4ad-3b7af36b0608 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:42.129690 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e1c6c-2690-43c3-a4ad-3b7af36b0608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:42.129966 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e1c6c-2690-43c3-a4ad-3b7af36b0608 admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:42.130696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e1c6c-2690-43c3-a4ad-3b7af36b0608 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:31:42.131004 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e1c6c-2690-43c3-a4ad-3b7af36b0608 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:42.133270 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e1c6c-2690-43c3-a4ad-3b7af36b0608 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:42.145830 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1036/2045] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:31:42.150723 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e1c6c-2690-43c3-a4ad-3b7af36b0608 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:42.152261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5cfb2-cade-416b-929e-c2181b257346 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:42.157132 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdda0a4-b594-4cf6-a981-45b7709c6932 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:42.157631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdda0a4-b594-4cf6-a981-45b7709c6932 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:42.157915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdda0a4-b594-4cf6-a981-45b7709c6932 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:42.158194 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdda0a4-b594-4cf6-a981-45b7709c6932 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:42.160864 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e1c6c-2690-43c3-a4ad-3b7af36b0608 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:42.166073 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5f767-f3c4-4055-809b-23a938d3d30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:42.166720 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5f767-f3c4-4055-809b-23a938d3d30e admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:42.167018 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5f767-f3c4-4055-809b-23a938d3d30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:42.167398 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5f767-f3c4-4055-809b-23a938d3d30e admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:42.168268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5f767-f3c4-4055-809b-23a938d3d30e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:42.169847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdda0a4-b594-4cf6-a981-45b7709c6932 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:42.170929 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5f767-f3c4-4055-809b-23a938d3d30e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:42.171418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdda0a4-b594-4cf6-a981-45b7709c6932 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:42.182911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1010/2046] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:42 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:31:42.201713 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1037/2047] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:42 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:31:42.206018 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdda0a4-b594-4cf6-a981-45b7709c6932 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:42.219422 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdda0a4-b594-4cf6-a981-45b7709c6932 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:42.277534 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:42.294365 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593b502-2480-4864-b1c9-8291107d3352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:42.295312 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593b502-2480-4864-b1c9-8291107d3352 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:42.295748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593b502-2480-4864-b1c9-8291107d3352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:42.305339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593b502-2480-4864-b1c9-8291107d3352 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:42.306843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593b502-2480-4864-b1c9-8291107d3352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:42.309597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593b502-2480-4864-b1c9-8291107d3352 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:42.336481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1011/2048] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:42 2022] POST /identity/v3/users => generated 297 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:42.340342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5f767-f3c4-4055-809b-23a938d3d30e admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:42.353935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5f767-f3c4-4055-809b-23a938d3d30e admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:42.359278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db67a7-211c-4d90-bddd-fbaea754f287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:42.359952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db67a7-211c-4d90-bddd-fbaea754f287 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:42.360302 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db67a7-211c-4d90-bddd-fbaea754f287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:42.360584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db67a7-211c-4d90-bddd-fbaea754f287 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:42.361386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db67a7-211c-4d90-bddd-fbaea754f287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:42.363087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db67a7-211c-4d90-bddd-fbaea754f287 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:42.367522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:42.382408 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1038/2049] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:42.385832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593b502-2480-4864-b1c9-8291107d3352 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:42.400016 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593b502-2480-4864-b1c9-8291107d3352 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:42.405747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4329cd-6334-4063-8f65-1a5ba05abb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:42.406394 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4329cd-6334-4063-8f65-1a5ba05abb87 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:42.406718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4329cd-6334-4063-8f65-1a5ba05abb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:42.406974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4329cd-6334-4063-8f65-1a5ba05abb87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:42.407952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4329cd-6334-4063-8f65-1a5ba05abb87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:42.409561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4329cd-6334-4063-8f65-1a5ba05abb87 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:42.422976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1012/2050] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:42.427424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db67a7-211c-4d90-bddd-fbaea754f287 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:42.440129 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db67a7-211c-4d90-bddd-fbaea754f287 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:42.445818 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf342a-0127-4c09-af00-4a2cc604d22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:42.446809 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf342a-0127-4c09-af00-4a2cc604d22d admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:42.447158 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf342a-0127-4c09-af00-4a2cc604d22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:42.447620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf342a-0127-4c09-af00-4a2cc604d22d admin admin] PATH_INFO: `/v3/projects/8c3bc3582aa7420a9a1fcccacff362bd/users/491d052f3d2947cea318b0753fffd1fa/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:42.463111 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf342a-0127-4c09-af00-4a2cc604d22d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c3bc3582aa7420a9a1fcccacff362bd, user_id=491d052f3d2947cea318b0753fffd1fa, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:42.466017 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf342a-0127-4c09-af00-4a2cc604d22d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:42.480751 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1039/2051] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:42 2022] PUT /identity/v3/projects/8c3bc3582aa7420a9a1fcccacff362bd/users/491d052f3d2947cea318b0753fffd1fa/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:42.486322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15b416-d7bb-4cdc-8975-61b8b31cb555 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:42.486618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15b416-d7bb-4cdc-8975-61b8b31cb555 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:42.486904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15b416-d7bb-4cdc-8975-61b8b31cb555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:42.505548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ce15b416-d7bb-4cdc-8975-61b8b31cb555 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:42.506815 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ce15b416-d7bb-4cdc-8975-61b8b31cb555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:42.570791 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1013/2052] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:42.582244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf342a-0127-4c09-af00-4a2cc604d22d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:42.584451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2dcf342a-0127-4c09-af00-4a2cc604d22d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:42.652569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf342a-0127-4c09-af00-4a2cc604d22d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:42.656455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433dc8d-7c2a-499a-b267-be9a045c2c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:42.657183 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433dc8d-7c2a-499a-b267-be9a045c2c05 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:42.668088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433dc8d-7c2a-499a-b267-be9a045c2c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:42.668088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433dc8d-7c2a-499a-b267-be9a045c2c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:42.668894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433dc8d-7c2a-499a-b267-be9a045c2c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:42.670615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433dc8d-7c2a-499a-b267-be9a045c2c05 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:42.686333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1040/2053] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:43.782364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15b416-d7bb-4cdc-8975-61b8b31cb555 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:43.784604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ce15b416-d7bb-4cdc-8975-61b8b31cb555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:43.799024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15b416-d7bb-4cdc-8975-61b8b31cb555 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:43.805784 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77290186-7d08-4bf3-a67c-74d5570853a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:43.806692 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77290186-7d08-4bf3-a67c-74d5570853a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:43.806981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77290186-7d08-4bf3-a67c-74d5570853a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:43.807261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77290186-7d08-4bf3-a67c-74d5570853a9 admin admin] PATH_INFO: `/v3/users/793271de95d3456abf405051f7caf516` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:43.808527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77290186-7d08-4bf3-a67c-74d5570853a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=793271de95d3456abf405051f7caf516)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:43.810507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77290186-7d08-4bf3-a67c-74d5570853a9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:43.869122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-77290186-7d08-4bf3-a67c-74d5570853a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '793271de95d3456abf405051f7caf516'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:43.887396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-77290186-7d08-4bf3-a67c-74d5570853a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '793271de95d3456abf405051f7caf516'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:43.888638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-77290186-7d08-4bf3-a67c-74d5570853a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '793271de95d3456abf405051f7caf516'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:43.889319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-77290186-7d08-4bf3-a67c-74d5570853a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '793271de95d3456abf405051f7caf516'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:43.902008 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-77290186-7d08-4bf3-a67c-74d5570853a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '793271de95d3456abf405051f7caf516'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:43.917552 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1014/2054] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:43 2022] DELETE /identity/v3/users/793271de95d3456abf405051f7caf516 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:43.921949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c61bb-d3d0-4b2b-a107-623f01e1179a None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:43.922445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c61bb-d3d0-4b2b-a107-623f01e1179a None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:43.922685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c61bb-d3d0-4b2b-a107-623f01e1179a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:43.946804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-0d4c61bb-d3d0-4b2b-a107-623f01e1179a None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:43.948296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0d4c61bb-d3d0-4b2b-a107-623f01e1179a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:44.051612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1041/2055] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:44.062235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77290186-7d08-4bf3-a67c-74d5570853a9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:44.064898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-77290186-7d08-4bf3-a67c-74d5570853a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:44.159050 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:44.184032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77290186-7d08-4bf3-a67c-74d5570853a9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:44.188783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e6128-ce1f-4652-8f48-9da543c0c3fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:44.189669 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e6128-ce1f-4652-8f48-9da543c0c3fb service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:44.190036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e6128-ce1f-4652-8f48-9da543c0c3fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:44.190385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e6128-ce1f-4652-8f48-9da543c0c3fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:44.203459 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e6128-ce1f-4652-8f48-9da543c0c3fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:44.205351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e6128-ce1f-4652-8f48-9da543c0c3fb service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:44.225721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1015/2056] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:44.599559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c61bb-d3d0-4b2b-a107-623f01e1179a None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:44.601987 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0d4c61bb-d3d0-4b2b-a107-623f01e1179a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:44.635550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c61bb-d3d0-4b2b-a107-623f01e1179a None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:44.645051 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b02fda-97ff-4dea-a702-96d234ffc83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:44.646290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b02fda-97ff-4dea-a702-96d234ffc83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:44.646888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b02fda-97ff-4dea-a702-96d234ffc83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:44.650344 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b02fda-97ff-4dea-a702-96d234ffc83f admin admin] PATH_INFO: `/v3/projects/48aded3a02ee452bb8be3ae636d33238` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:44.652861 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b02fda-97ff-4dea-a702-96d234ffc83f admin admin] RBAC: Authorizing `identity:delete_project(project_id=48aded3a02ee452bb8be3ae636d33238)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:44.656036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b02fda-97ff-4dea-a702-96d234ffc83f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:44.762559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-75b02fda-97ff-4dea-a702-96d234ffc83f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48aded3a02ee452bb8be3ae636d33238', 'request_id': 'req-75b02fda-97ff-4dea-a702-96d234ffc83f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:44.769575 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-75b02fda-97ff-4dea-a702-96d234ffc83f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48aded3a02ee452bb8be3ae636d33238', 'request_id': 'req-75b02fda-97ff-4dea-a702-96d234ffc83f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:44.780347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-75b02fda-97ff-4dea-a702-96d234ffc83f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48aded3a02ee452bb8be3ae636d33238', 'request_id': 'req-75b02fda-97ff-4dea-a702-96d234ffc83f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:44.801867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:44.808354 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-75b02fda-97ff-4dea-a702-96d234ffc83f admin admin] The token cache is being invalidate because project 48aded3a02ee452bb8be3ae636d33238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:44.808698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-75b02fda-97ff-4dea-a702-96d234ffc83f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:44.810695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-75b02fda-97ff-4dea-a702-96d234ffc83f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:44.812523 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1042/2057] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:44 2022] DELETE /identity/v3/projects/48aded3a02ee452bb8be3ae636d33238 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:44.817431 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e6128-ce1f-4652-8f48-9da543c0c3fb service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:44.833719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e6128-ce1f-4652-8f48-9da543c0c3fb service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:44.895476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b02fda-97ff-4dea-a702-96d234ffc83f admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:44.897585 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-75b02fda-97ff-4dea-a702-96d234ffc83f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:44.905638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e8b92-a98d-4ac4-8f32-e7ea8838afce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:44.907040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e8b92-a98d-4ac4-8f32-e7ea8838afce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:44.907422 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e8b92-a98d-4ac4-8f32-e7ea8838afce admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:44.907796 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e8b92-a98d-4ac4-8f32-e7ea8838afce admin admin] PATH_INFO: `/v3/users/7b0db8536cbf4042961d12f4064aab51` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:44.909736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e8b92-a98d-4ac4-8f32-e7ea8838afce admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b0db8536cbf4042961d12f4064aab51)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:44.912241 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e8b92-a98d-4ac4-8f32-e7ea8838afce admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:44.970424 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a48e8b92-a98d-4ac4-8f32-e7ea8838afce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b0db8536cbf4042961d12f4064aab51'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:44.971692 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b02fda-97ff-4dea-a702-96d234ffc83f admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:44.978391 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71810d2-48d1-4b82-a8a7-a035e1593a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:44.979451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71810d2-48d1-4b82-a8a7-a035e1593a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:44.979739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71810d2-48d1-4b82-a8a7-a035e1593a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:44.980020 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71810d2-48d1-4b82-a8a7-a035e1593a6f admin admin] PATH_INFO: `/v3/users/c51857e9cf5c461789bae58821e7b7a5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:44.981305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71810d2-48d1-4b82-a8a7-a035e1593a6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c51857e9cf5c461789bae58821e7b7a5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:44.983886 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71810d2-48d1-4b82-a8a7-a035e1593a6f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:44.987202 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a48e8b92-a98d-4ac4-8f32-e7ea8838afce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b0db8536cbf4042961d12f4064aab51'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:44.987985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a48e8b92-a98d-4ac4-8f32-e7ea8838afce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b0db8536cbf4042961d12f4064aab51'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:44.988605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a48e8b92-a98d-4ac4-8f32-e7ea8838afce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b0db8536cbf4042961d12f4064aab51'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:44.996528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a48e8b92-a98d-4ac4-8f32-e7ea8838afce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b0db8536cbf4042961d12f4064aab51'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:45.016723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1016/2058] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:44 2022] DELETE /identity/v3/users/7b0db8536cbf4042961d12f4064aab51 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:45.053867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-e71810d2-48d1-4b82-a8a7-a035e1593a6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c51857e9cf5c461789bae58821e7b7a5'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:45.073915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-e71810d2-48d1-4b82-a8a7-a035e1593a6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c51857e9cf5c461789bae58821e7b7a5'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:45.075096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-e71810d2-48d1-4b82-a8a7-a035e1593a6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c51857e9cf5c461789bae58821e7b7a5'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:45.088444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-e71810d2-48d1-4b82-a8a7-a035e1593a6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c51857e9cf5c461789bae58821e7b7a5'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:45.089295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-e71810d2-48d1-4b82-a8a7-a035e1593a6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c51857e9cf5c461789bae58821e7b7a5'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:45.114775 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e371166-59f7-4fff-a501-cfd44f4c1f5f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:45.115173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e371166-59f7-4fff-a501-cfd44f4c1f5f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:45.115606 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e371166-59f7-4fff-a501-cfd44f4c1f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:45.117291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1043/2059] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:44 2022] DELETE /identity/v3/users/c51857e9cf5c461789bae58821e7b7a5 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:31:45.136618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1e371166-59f7-4fff-a501-cfd44f4c1f5f None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:45.138544 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1e371166-59f7-4fff-a501-cfd44f4c1f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:45.219552 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:45.235174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1017/2060] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:45.250291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71810d2-48d1-4b82-a8a7-a035e1593a6f admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:45.252770 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e71810d2-48d1-4b82-a8a7-a035e1593a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:45.359458 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71810d2-48d1-4b82-a8a7-a035e1593a6f admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:45.364338 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fc63e-1b4d-4919-9121-cb83c32bafef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:45.365067 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fc63e-1b4d-4919-9121-cb83c32bafef service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:45.365646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fc63e-1b4d-4919-9121-cb83c32bafef service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:45.365979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fc63e-1b4d-4919-9121-cb83c32bafef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:45.386089 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fc63e-1b4d-4919-9121-cb83c32bafef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:45.387660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fc63e-1b4d-4919-9121-cb83c32bafef service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:45.400289 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1044/2061] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:45.416688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e371166-59f7-4fff-a501-cfd44f4c1f5f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:45.418283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1e371166-59f7-4fff-a501-cfd44f4c1f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:45.457228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e371166-59f7-4fff-a501-cfd44f4c1f5f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:45.463974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba8f31-131a-4ab2-a59e-ca68461d1715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:45.466832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba8f31-131a-4ab2-a59e-ca68461d1715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:45.467215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba8f31-131a-4ab2-a59e-ca68461d1715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:45.467594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba8f31-131a-4ab2-a59e-ca68461d1715 admin admin] PATH_INFO: `/v3/projects/1f42d8478cce4b3687d121e6b7277d27` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:45.469356 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba8f31-131a-4ab2-a59e-ca68461d1715 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f42d8478cce4b3687d121e6b7277d27)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:45.471385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba8f31-131a-4ab2-a59e-ca68461d1715 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:45.551896 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2eba8f31-131a-4ab2-a59e-ca68461d1715 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f42d8478cce4b3687d121e6b7277d27', 'request_id': 'req-2eba8f31-131a-4ab2-a59e-ca68461d1715'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:45.567442 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2eba8f31-131a-4ab2-a59e-ca68461d1715 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f42d8478cce4b3687d121e6b7277d27', 'request_id': 'req-2eba8f31-131a-4ab2-a59e-ca68461d1715'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:45.575968 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2eba8f31-131a-4ab2-a59e-ca68461d1715 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f42d8478cce4b3687d121e6b7277d27', 'request_id': 'req-2eba8f31-131a-4ab2-a59e-ca68461d1715'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:45.602848 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2eba8f31-131a-4ab2-a59e-ca68461d1715 admin admin] The token cache is being invalidate because project 1f42d8478cce4b3687d121e6b7277d27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:45.613258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2eba8f31-131a-4ab2-a59e-ca68461d1715 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:45.616368 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2eba8f31-131a-4ab2-a59e-ca68461d1715 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:45.621183 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1018/2062] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:45 2022] DELETE /identity/v3/projects/1f42d8478cce4b3687d121e6b7277d27 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:45.623300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be00181-88c4-45b9-be78-ac2ac23a570b None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:45.623597 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be00181-88c4-45b9-be78-ac2ac23a570b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:45.623883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be00181-88c4-45b9-be78-ac2ac23a570b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:45.670989 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-6be00181-88c4-45b9-be78-ac2ac23a570b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:45.675037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-6be00181-88c4-45b9-be78-ac2ac23a570b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:45.713881 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:45.784505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1045/2063] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:45.793011 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026158d2-1ecf-489a-a007-7f6dd96e0a40 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:45.793427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026158d2-1ecf-489a-a007-7f6dd96e0a40 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:45.793805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026158d2-1ecf-489a-a007-7f6dd96e0a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:45.819356 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-026158d2-1ecf-489a-a007-7f6dd96e0a40 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:45.821434 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-026158d2-1ecf-489a-a007-7f6dd96e0a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:45.838019 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1019/2064] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:45.839266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be00181-88c4-45b9-be78-ac2ac23a570b None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:45.856527 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be00181-88c4-45b9-be78-ac2ac23a570b None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:45.866455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d435c24-9f7e-420b-af3d-a4dad5e6aafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:45.869210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d435c24-9f7e-420b-af3d-a4dad5e6aafe admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:45.869598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d435c24-9f7e-420b-af3d-a4dad5e6aafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:45.869955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d435c24-9f7e-420b-af3d-a4dad5e6aafe admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:45.871020 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d435c24-9f7e-420b-af3d-a4dad5e6aafe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:31:45.871216 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d435c24-9f7e-420b-af3d-a4dad5e6aafe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:45.872954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d435c24-9f7e-420b-af3d-a4dad5e6aafe admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:45.888415 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1046/2065] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:31:45.892766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026158d2-1ecf-489a-a007-7f6dd96e0a40 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:45.906611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026158d2-1ecf-489a-a007-7f6dd96e0a40 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:45.919132 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47750f9d-0dbf-4971-a459-9663bda18a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:45.920091 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47750f9d-0dbf-4971-a459-9663bda18a93 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:45.920453 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47750f9d-0dbf-4971-a459-9663bda18a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:45.920831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47750f9d-0dbf-4971-a459-9663bda18a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:45.921659 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47750f9d-0dbf-4971-a459-9663bda18a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:45.934876 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47750f9d-0dbf-4971-a459-9663bda18a93 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:45.972535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1020/2066] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:45 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:45.973994 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d435c24-9f7e-420b-af3d-a4dad5e6aafe admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:45.990995 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d435c24-9f7e-420b-af3d-a4dad5e6aafe admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:46.076513 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce863269-53fd-4b33-bad6-4b2cd8dc85fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:46.077974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce863269-53fd-4b33-bad6-4b2cd8dc85fc admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:46.078269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce863269-53fd-4b33-bad6-4b2cd8dc85fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:46.078548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce863269-53fd-4b33-bad6-4b2cd8dc85fc admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:46.084018 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce863269-53fd-4b33-bad6-4b2cd8dc85fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:46.084018 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce863269-53fd-4b33-bad6-4b2cd8dc85fc admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:46.086140 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47750f9d-0dbf-4971-a459-9663bda18a93 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:46.088315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-47750f9d-0dbf-4971-a459-9663bda18a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:46.116053 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47750f9d-0dbf-4971-a459-9663bda18a93 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:46.121955 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce6236-57c2-4437-babb-7c3df18167a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:46.123634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce6236-57c2-4437-babb-7c3df18167a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:46.124025 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce6236-57c2-4437-babb-7c3df18167a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:46.124810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce6236-57c2-4437-babb-7c3df18167a1 admin admin] PATH_INFO: `/v3/projects/cbe88e8efc054e9da5ac6b6790cb3a66` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:46.126498 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce6236-57c2-4437-babb-7c3df18167a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbe88e8efc054e9da5ac6b6790cb3a66)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:46.128812 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce6236-57c2-4437-babb-7c3df18167a1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:46.144496 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1047/2067] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:45 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:46.145435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce863269-53fd-4b33-bad6-4b2cd8dc85fc admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:46.170400 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce863269-53fd-4b33-bad6-4b2cd8dc85fc admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:46.186259 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf4600-f446-4d98-b13d-5e548800feee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:46.186945 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf4600-f446-4d98-b13d-5e548800feee admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:46.187227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf4600-f446-4d98-b13d-5e548800feee admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:46.187503 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf4600-f446-4d98-b13d-5e548800feee admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:46.188309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf4600-f446-4d98-b13d-5e548800feee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:46.191657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8fce6236-57c2-4437-babb-7c3df18167a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbe88e8efc054e9da5ac6b6790cb3a66', 'request_id': 'req-8fce6236-57c2-4437-babb-7c3df18167a1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:46.194045 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf4600-f446-4d98-b13d-5e548800feee admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:46.205590 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8fce6236-57c2-4437-babb-7c3df18167a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbe88e8efc054e9da5ac6b6790cb3a66', 'request_id': 'req-8fce6236-57c2-4437-babb-7c3df18167a1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:46.218447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8fce6236-57c2-4437-babb-7c3df18167a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbe88e8efc054e9da5ac6b6790cb3a66', 'request_id': 'req-8fce6236-57c2-4437-babb-7c3df18167a1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:46.232851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1048/2068] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:46.233859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf4600-f446-4d98-b13d-5e548800feee admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:46.235498 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8fce6236-57c2-4437-babb-7c3df18167a1 admin admin] The token cache is being invalidate because project cbe88e8efc054e9da5ac6b6790cb3a66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:46.236149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8fce6236-57c2-4437-babb-7c3df18167a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:46.236942 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8fce6236-57c2-4437-babb-7c3df18167a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:46.242825 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1021/2069] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:46 2022] DELETE /identity/v3/projects/cbe88e8efc054e9da5ac6b6790cb3a66 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:46.244554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b051c04-9f7b-4659-ad79-9b4a4351e52c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:46.244963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b051c04-9f7b-4659-ad79-9b4a4351e52c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:46.246268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b051c04-9f7b-4659-ad79-9b4a4351e52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:46.253264 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-baaf4600-f446-4d98-b13d-5e548800feee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:46.284335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf4600-f446-4d98-b13d-5e548800feee admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:46.311604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4b051c04-9f7b-4659-ad79-9b4a4351e52c None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:46.316637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4b051c04-9f7b-4659-ad79-9b4a4351e52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:46.388738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:46.438044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a9270-d6ad-4216-a0ac-aeba6f4a05de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:46.439035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a9270-d6ad-4216-a0ac-aeba6f4a05de admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:46.439985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a9270-d6ad-4216-a0ac-aeba6f4a05de admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:46.440279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a9270-d6ad-4216-a0ac-aeba6f4a05de admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:46.444090 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a9270-d6ad-4216-a0ac-aeba6f4a05de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:46.446926 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a9270-d6ad-4216-a0ac-aeba6f4a05de admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:46.458416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1022/2070] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:46.460737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a338f-cca9-4fea-84cd-a571191bf2e6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:46.461281 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a338f-cca9-4fea-84cd-a571191bf2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:46.461579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a338f-cca9-4fea-84cd-a571191bf2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:46.473045 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1049/2071] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:46.484869 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a9270-d6ad-4216-a0ac-aeba6f4a05de admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:46.530510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a9270-d6ad-4216-a0ac-aeba6f4a05de admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:46.532838 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0b6a338f-cca9-4fea-84cd-a571191bf2e6 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:46.535167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0b6a338f-cca9-4fea-84cd-a571191bf2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:46.546064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed2af0-e48c-4f93-87a5-290dd13e033d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:46.547411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2af0-e48c-4f93-87a5-290dd13e033d admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:46.547850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2af0-e48c-4f93-87a5-290dd13e033d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:46.548492 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2af0-e48c-4f93-87a5-290dd13e033d admin admin] PATH_INFO: `/v3/projects/210471bf1a9a494884ba685b7ccb5f67/users/f1e4c061613b4c1fa53b4ffbbb4a8d89/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:46.565711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1023/2072] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:46.567218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a338f-cca9-4fea-84cd-a571191bf2e6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:46.592249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a338f-cca9-4fea-84cd-a571191bf2e6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:46.593449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed2af0-e48c-4f93-87a5-290dd13e033d admin admin] RBAC: Authorizing `identity:create_grant(project_id=210471bf1a9a494884ba685b7ccb5f67, user_id=f1e4c061613b4c1fa53b4ffbbb4a8d89, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:46.596326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed2af0-e48c-4f93-87a5-290dd13e033d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:46.610379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23209606-ab4e-4e0a-b7a6-abc73e0564b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:46.612865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23209606-ab4e-4e0a-b7a6-abc73e0564b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:46.613750 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23209606-ab4e-4e0a-b7a6-abc73e0564b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:46.615689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23209606-ab4e-4e0a-b7a6-abc73e0564b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:46.617304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23209606-ab4e-4e0a-b7a6-abc73e0564b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:31:46.618849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:46.620445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23209606-ab4e-4e0a-b7a6-abc73e0564b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:46.622752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23209606-ab4e-4e0a-b7a6-abc73e0564b9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:46.625573 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1050/2073] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:46 2022] PUT /identity/v3/projects/210471bf1a9a494884ba685b7ccb5f67/users/f1e4c061613b4c1fa53b4ffbbb4a8d89/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:46.639902 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8d57d-aab9-4047-b7de-dcdc4eea1916 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:46.640331 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8d57d-aab9-4047-b7de-dcdc4eea1916 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:46.640700 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8d57d-aab9-4047-b7de-dcdc4eea1916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:46.665255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1024/2074] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:31:46.668440 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23209606-ab4e-4e0a-b7a6-abc73e0564b9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:46.685043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23209606-ab4e-4e0a-b7a6-abc73e0564b9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:46.691894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-f3e8d57d-aab9-4047-b7de-dcdc4eea1916 None None] MFA Rules not processed for user `f1e4c061613b4c1fa53b4ffbbb4a8d89`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:46.693514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f3e8d57d-aab9-4047-b7de-dcdc4eea1916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:46.800833 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2926c1-8e78-4828-ac19-a510f81a9ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:46.802183 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2926c1-8e78-4828-ac19-a510f81a9ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:46.802670 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2926c1-8e78-4828-ac19-a510f81a9ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:46.803232 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2926c1-8e78-4828-ac19-a510f81a9ef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:46.804114 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2926c1-8e78-4828-ac19-a510f81a9ef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:46.808324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2926c1-8e78-4828-ac19-a510f81a9ef6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:46.832871 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1051/2075] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:46.836305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8d57d-aab9-4047-b7de-dcdc4eea1916 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:46.861692 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8d57d-aab9-4047-b7de-dcdc4eea1916 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:46.869401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44ae39-c22d-4146-a77e-56a9e38e6e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:46.873180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1025/2076] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:46 2022] POST /identity/v3/projects => generated 369 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:31:46.875568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2926c1-8e78-4828-ac19-a510f81a9ef6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:46.878670 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44ae39-c22d-4146-a77e-56a9e38e6e84 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:46.879126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44ae39-c22d-4146-a77e-56a9e38e6e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:46.879443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44ae39-c22d-4146-a77e-56a9e38e6e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:46.880820 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44ae39-c22d-4146-a77e-56a9e38e6e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:46.884702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44ae39-c22d-4146-a77e-56a9e38e6e84 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:46.954704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1052/2077] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:46 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:46.960178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44ae39-c22d-4146-a77e-56a9e38e6e84 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:47.033428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2926c1-8e78-4828-ac19-a510f81a9ef6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:47.110256 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44ae39-c22d-4146-a77e-56a9e38e6e84 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:47.119414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babc310-fe36-4564-a852-f1646bf45b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:47.120218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babc310-fe36-4564-a852-f1646bf45b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:47.120541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babc310-fe36-4564-a852-f1646bf45b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:47.120984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babc310-fe36-4564-a852-f1646bf45b31 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:47.121871 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6babc310-fe36-4564-a852-f1646bf45b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:47.125213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6babc310-fe36-4564-a852-f1646bf45b31 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:47.139908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:47.156427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec70de2-56e1-4639-98bc-0432f07fb511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:47.158771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec70de2-56e1-4639-98bc-0432f07fb511 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:47.159420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec70de2-56e1-4639-98bc-0432f07fb511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:47.159842 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1053/2078] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:46 2022] POST /identity/v3/users => generated 295 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:31:47.160322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec70de2-56e1-4639-98bc-0432f07fb511 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:47.161302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec70de2-56e1-4639-98bc-0432f07fb511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:47.164312 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec70de2-56e1-4639-98bc-0432f07fb511 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:47.166189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babc310-fe36-4564-a852-f1646bf45b31 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:47.192410 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1026/2079] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:46 2022] POST /identity/v3/users => generated 285 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:47.198030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec70de2-56e1-4639-98bc-0432f07fb511 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:47.213282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babc310-fe36-4564-a852-f1646bf45b31 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:47.214000 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec70de2-56e1-4639-98bc-0432f07fb511 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:47.222822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034c5cc-b469-420e-86b7-aea2661b072b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:47.223320 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034c5cc-b469-420e-86b7-aea2661b072b admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:47.223692 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034c5cc-b469-420e-86b7-aea2661b072b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:47.224040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034c5cc-b469-420e-86b7-aea2661b072b admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:47.225092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034c5cc-b469-420e-86b7-aea2661b072b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:47.231362 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba582f-e6e6-41dd-b148-cda6f91eb84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:47.231362 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba582f-e6e6-41dd-b148-cda6f91eb84e admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:47.231362 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba582f-e6e6-41dd-b148-cda6f91eb84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:47.232036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034c5cc-b469-420e-86b7-aea2661b072b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:47.232222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba582f-e6e6-41dd-b148-cda6f91eb84e admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:47.232222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba582f-e6e6-41dd-b148-cda6f91eb84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:47.234281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba582f-e6e6-41dd-b148-cda6f91eb84e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:47.252270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1027/2080] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:31:47.253505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034c5cc-b469-420e-86b7-aea2661b072b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:47.275548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034c5cc-b469-420e-86b7-aea2661b072b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:47.280804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1054/2081] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:47.287404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba582f-e6e6-41dd-b148-cda6f91eb84e admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:47.288203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b137f-600b-4a44-9e9e-97548fc1d972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:47.288945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b137f-600b-4a44-9e9e-97548fc1d972 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:47.289320 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b137f-600b-4a44-9e9e-97548fc1d972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:47.290046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b137f-600b-4a44-9e9e-97548fc1d972 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:47.290869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b137f-600b-4a44-9e9e-97548fc1d972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:47.295374 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b137f-600b-4a44-9e9e-97548fc1d972 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:47.308732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba582f-e6e6-41dd-b148-cda6f91eb84e admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:47.314290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5aaed7-9277-4dec-9ad2-2aebe15d4849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:47.315329 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5aaed7-9277-4dec-9ad2-2aebe15d4849 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:47.315714 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5aaed7-9277-4dec-9ad2-2aebe15d4849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:47.316191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5aaed7-9277-4dec-9ad2-2aebe15d4849 admin admin] PATH_INFO: `/v3/projects/5d57435b3df34818a0e3c62339ff91b2/users/a07fa4a6e5ba41929c18ab5433451c8d/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:47.337743 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5aaed7-9277-4dec-9ad2-2aebe15d4849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d57435b3df34818a0e3c62339ff91b2, user_id=a07fa4a6e5ba41929c18ab5433451c8d, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:47.338286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1028/2082] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:47.339374 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b137f-600b-4a44-9e9e-97548fc1d972 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:47.342309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5aaed7-9277-4dec-9ad2-2aebe15d4849 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:47.363792 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b137f-600b-4a44-9e9e-97548fc1d972 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:47.367668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1055/2083] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:47 2022] PUT /identity/v3/projects/5d57435b3df34818a0e3c62339ff91b2/users/a07fa4a6e5ba41929c18ab5433451c8d/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:47.372617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5aaed7-9277-4dec-9ad2-2aebe15d4849 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:47.418677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:47.476932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5aaed7-9277-4dec-9ad2-2aebe15d4849 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:47.485958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f816d16-04d3-4a1b-a570-3f6f2e4e798d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:47.487405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f816d16-04d3-4a1b-a570-3f6f2e4e798d admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:47.487695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f816d16-04d3-4a1b-a570-3f6f2e4e798d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:47.488129 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f816d16-04d3-4a1b-a570-3f6f2e4e798d admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:47.489032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb15c85-8d6f-4a10-a8d6-fa2c47e14883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:47.489554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f816d16-04d3-4a1b-a570-3f6f2e4e798d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:47.490337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb15c85-8d6f-4a10-a8d6-fa2c47e14883 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:47.491132 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb15c85-8d6f-4a10-a8d6-fa2c47e14883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:47.492065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb15c85-8d6f-4a10-a8d6-fa2c47e14883 admin admin] PATH_INFO: `/v3/projects/1e602a9a23df4fbd80447123544cfe58/users/68b8d711f6b1463eb4449699d537c4ea/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:47.493375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f816d16-04d3-4a1b-a570-3f6f2e4e798d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:47.509435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1056/2084] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:47.514181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f816d16-04d3-4a1b-a570-3f6f2e4e798d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:47.514336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb15c85-8d6f-4a10-a8d6-fa2c47e14883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e602a9a23df4fbd80447123544cfe58, user_id=68b8d711f6b1463eb4449699d537c4ea, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:47.517389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb15c85-8d6f-4a10-a8d6-fa2c47e14883 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:47.539585 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f816d16-04d3-4a1b-a570-3f6f2e4e798d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:47.555524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1029/2085] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:47 2022] PUT /identity/v3/projects/1e602a9a23df4fbd80447123544cfe58/users/68b8d711f6b1463eb4449699d537c4ea/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:47.557199 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45d22f-8afc-4ce8-a850-907f36be4705 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:47.557571 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45d22f-8afc-4ce8-a850-907f36be4705 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:47.558268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45d22f-8afc-4ce8-a850-907f36be4705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:47.604557 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5a45d22f-8afc-4ce8-a850-907f36be4705 None None] MFA Rules not processed for user `68b8d711f6b1463eb4449699d537c4ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:47.605920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5a45d22f-8afc-4ce8-a850-907f36be4705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:47.710958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1030/2086] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:47.727238 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d08f2-0a76-45f1-80df-c29a1c51e26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:47.727498 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45d22f-8afc-4ce8-a850-907f36be4705 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:47.728701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d08f2-0a76-45f1-80df-c29a1c51e26a admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:47.728995 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d08f2-0a76-45f1-80df-c29a1c51e26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:47.729656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d08f2-0a76-45f1-80df-c29a1c51e26a admin admin] PATH_INFO: `/v3/projects/5d57435b3df34818a0e3c62339ff91b2/users/a07fa4a6e5ba41929c18ab5433451c8d/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:47.729949 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5a45d22f-8afc-4ce8-a850-907f36be4705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:47.732280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d08f2-0a76-45f1-80df-c29a1c51e26a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d57435b3df34818a0e3c62339ff91b2, user_id=a07fa4a6e5ba41929c18ab5433451c8d, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:47.735177 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d08f2-0a76-45f1-80df-c29a1c51e26a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:47.772369 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1057/2087] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:47 2022] PUT /identity/v3/projects/5d57435b3df34818a0e3c62339ff91b2/users/a07fa4a6e5ba41929c18ab5433451c8d/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:47.784362 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f981e05-5b9c-4673-b11b-e7eaaaaafe8c None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:47.784944 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f981e05-5b9c-4673-b11b-e7eaaaaafe8c None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:47.785257 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f981e05-5b9c-4673-b11b-e7eaaaaafe8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:47.806945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:47.819635 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-6f981e05-5b9c-4673-b11b-e7eaaaaafe8c None None] MFA Rules not processed for user `a07fa4a6e5ba41929c18ab5433451c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:47.821222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-6f981e05-5b9c-4673-b11b-e7eaaaaafe8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:47.854284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45d22f-8afc-4ce8-a850-907f36be4705 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:47.948071 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1058/2088] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:47.954541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f981e05-5b9c-4673-b11b-e7eaaaaafe8c None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:47.954729 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396b80d-7e2d-4b70-a207-19b4d7252681 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:47.955298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396b80d-7e2d-4b70-a207-19b4d7252681 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:47.955574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396b80d-7e2d-4b70-a207-19b4d7252681 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:47.955861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396b80d-7e2d-4b70-a207-19b4d7252681 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:47.957913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-6f981e05-5b9c-4673-b11b-e7eaaaaafe8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:47.980969 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f981e05-5b9c-4673-b11b-e7eaaaaafe8c None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:47.997335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27080b-ebaa-4b03-b78a-68916eb443be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:47.997335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27080b-ebaa-4b03-b78a-68916eb443be service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:47.997631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27080b-ebaa-4b03-b78a-68916eb443be service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:47.997869 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27080b-ebaa-4b03-b78a-68916eb443be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:48.050595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396b80d-7e2d-4b70-a207-19b4d7252681 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:48.052065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396b80d-7e2d-4b70-a207-19b4d7252681 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:48.061914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:48.080196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1031/2089] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:47 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:31:48.091247 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27080b-ebaa-4b03-b78a-68916eb443be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:48.092873 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27080b-ebaa-4b03-b78a-68916eb443be service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:48.118789 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1059/2090] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:48.161270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001ceb9-9590-4bc5-9038-b17069ad0174 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:48.161469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001ceb9-9590-4bc5-9038-b17069ad0174 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:48.161829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001ceb9-9590-4bc5-9038-b17069ad0174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:48.214796 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b001ceb9-9590-4bc5-9038-b17069ad0174 None None] MFA Rules not processed for user `491d052f3d2947cea318b0753fffd1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:48.216812 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b001ceb9-9590-4bc5-9038-b17069ad0174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:48.378402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1032/2091] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:48.389977 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460273e-caf5-4753-95d7-c8f2293c3e6e None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:48.390270 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460273e-caf5-4753-95d7-c8f2293c3e6e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:48.390555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460273e-caf5-4753-95d7-c8f2293c3e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:48.413569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-3460273e-caf5-4753-95d7-c8f2293c3e6e None None] MFA Rules not processed for user `491d052f3d2947cea318b0753fffd1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:48.415104 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3460273e-caf5-4753-95d7-c8f2293c3e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:48.427625 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1060/2092] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:48.434999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001ceb9-9590-4bc5-9038-b17069ad0174 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:48.436424 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b001ceb9-9590-4bc5-9038-b17069ad0174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:48.501076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:48.556584 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001ceb9-9590-4bc5-9038-b17069ad0174 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:48.564114 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591b8ba-1b32-4ad4-aa44-7b8f58d7ce5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:48.564430 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591b8ba-1b32-4ad4-aa44-7b8f58d7ce5d service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:48.564635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591b8ba-1b32-4ad4-aa44-7b8f58d7ce5d service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:48.564921 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591b8ba-1b32-4ad4-aa44-7b8f58d7ce5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:48.574951 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591b8ba-1b32-4ad4-aa44-7b8f58d7ce5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:48.586286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591b8ba-1b32-4ad4-aa44-7b8f58d7ce5d service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:48.612684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1033/2093] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:49.626157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc81ff-fb2c-4230-9070-5e903db320cf None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:49.626533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc81ff-fb2c-4230-9070-5e903db320cf None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:49.626632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc81ff-fb2c-4230-9070-5e903db320cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:49.648098 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-cadc81ff-fb2c-4230-9070-5e903db320cf None None] MFA Rules not processed for user `491d052f3d2947cea318b0753fffd1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:49.649926 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cadc81ff-fb2c-4230-9070-5e903db320cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:49.666092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38790b-eab2-4365-9281-484ab07489a0 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:49.666600 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38790b-eab2-4365-9281-484ab07489a0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:49.666898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38790b-eab2-4365-9281-484ab07489a0 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:49.671582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1034/2094] 10.222.0.42 () {54 vars in 921 bytes} [Tue Jul 19 15:31:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:31:49.692342 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e5a17-8e5c-46e3-983f-4c45f294e7c6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:49.692883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e5a17-8e5c-46e3-983f-4c45f294e7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:49.693453 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e5a17-8e5c-46e3-983f-4c45f294e7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:49.694558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1061/2095] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:49.751141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-752e5a17-8e5c-46e3-983f-4c45f294e7c6 None None] MFA Rules not processed for user `6002f32936944b0db6fd04f3851836db`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:49.754476 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-752e5a17-8e5c-46e3-983f-4c45f294e7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:49.877490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc81ff-fb2c-4230-9070-5e903db320cf None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:49.879318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cadc81ff-fb2c-4230-9070-5e903db320cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:49.962200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1035/2096] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Jul 19 15:31:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:50.074017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e5a17-8e5c-46e3-983f-4c45f294e7c6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:50.075514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-752e5a17-8e5c-46e3-983f-4c45f294e7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:50.094871 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc81ff-fb2c-4230-9070-5e903db320cf None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:50.113599 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e5a17-8e5c-46e3-983f-4c45f294e7c6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:50.116965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e29c50-d058-4f66-aeb1-7c1259049313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:50.117992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e29c50-d058-4f66-aeb1-7c1259049313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:50.118870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e29c50-d058-4f66-aeb1-7c1259049313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:50.119640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e29c50-d058-4f66-aeb1-7c1259049313 admin admin] PATH_INFO: `/v3/users/9e54c22586b74506b3246eab8de57807` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:50.123605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e29c50-d058-4f66-aeb1-7c1259049313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e54c22586b74506b3246eab8de57807)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:50.130493 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e29c50-d058-4f66-aeb1-7c1259049313 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:50.133262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b50856-44f6-495e-a7e7-e6d703f03d86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:50.135942 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b50856-44f6-495e-a7e7-e6d703f03d86 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:50.137898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b50856-44f6-495e-a7e7-e6d703f03d86 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:50.138205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b50856-44f6-495e-a7e7-e6d703f03d86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:50.170874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b50856-44f6-495e-a7e7-e6d703f03d86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:50.179464 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b50856-44f6-495e-a7e7-e6d703f03d86 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:50.246113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1036/2097] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:50.283182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-83e29c50-d058-4f66-aeb1-7c1259049313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e54c22586b74506b3246eab8de57807'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:50.301295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-83e29c50-d058-4f66-aeb1-7c1259049313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e54c22586b74506b3246eab8de57807'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:50.302351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-83e29c50-d058-4f66-aeb1-7c1259049313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e54c22586b74506b3246eab8de57807'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:50.315853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-83e29c50-d058-4f66-aeb1-7c1259049313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e54c22586b74506b3246eab8de57807'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:50.316560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-83e29c50-d058-4f66-aeb1-7c1259049313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e54c22586b74506b3246eab8de57807'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:50.335982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1062/2098] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:49 2022] DELETE /identity/v3/users/9e54c22586b74506b3246eab8de57807 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:31:50.722049 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b50856-44f6-495e-a7e7-e6d703f03d86 service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:50.737772 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b50856-44f6-495e-a7e7-e6d703f03d86 service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:50.821824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:50.822704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:50.823066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:50.823396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85 admin admin] PATH_INFO: `/v3/projects/12aebc5f34a54d6699a2efc6d3a1715b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:50.824940 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12aebc5f34a54d6699a2efc6d3a1715b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:50.827298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:50.890524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:50.900950 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12aebc5f34a54d6699a2efc6d3a1715b', 'request_id': 'req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:50.915810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12aebc5f34a54d6699a2efc6d3a1715b', 'request_id': 'req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:50.926406 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12aebc5f34a54d6699a2efc6d3a1715b', 'request_id': 'req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:50.956433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85 admin admin] The token cache is being invalidate because project 12aebc5f34a54d6699a2efc6d3a1715b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:50.956938 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:50.958251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:50.961275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1037/2099] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:50 2022] DELETE /identity/v3/projects/12aebc5f34a54d6699a2efc6d3a1715b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:50.965732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e29c50-d058-4f66-aeb1-7c1259049313 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:50.967008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-83e29c50-d058-4f66-aeb1-7c1259049313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:50.986997 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e29c50-d058-4f66-aeb1-7c1259049313 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:51.086858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:51.138103 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d046a-17e6-4309-bf9e-a65a888f7e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:51.139293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d046a-17e6-4309-bf9e-a65a888f7e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:51.140787 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d046a-17e6-4309-bf9e-a65a888f7e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:51.141275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d046a-17e6-4309-bf9e-a65a888f7e49 admin admin] PATH_INFO: `/v3/users/99f04eb288ed4f7f86f19dceca276d0c` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:51.144042 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d046a-17e6-4309-bf9e-a65a888f7e49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99f04eb288ed4f7f86f19dceca276d0c)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:51.146194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d046a-17e6-4309-bf9e-a65a888f7e49 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:51.281985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-121d046a-17e6-4309-bf9e-a65a888f7e49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99f04eb288ed4f7f86f19dceca276d0c'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:51.298293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-121d046a-17e6-4309-bf9e-a65a888f7e49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99f04eb288ed4f7f86f19dceca276d0c'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:51.299353 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-121d046a-17e6-4309-bf9e-a65a888f7e49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99f04eb288ed4f7f86f19dceca276d0c'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:51.310621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-121d046a-17e6-4309-bf9e-a65a888f7e49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99f04eb288ed4f7f86f19dceca276d0c'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:51.312465 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-121d046a-17e6-4309-bf9e-a65a888f7e49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99f04eb288ed4f7f86f19dceca276d0c'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:51.328741 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1063/2100] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:50 2022] DELETE /identity/v3/users/99f04eb288ed4f7f86f19dceca276d0c => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:51.582287 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:51.584212 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:51.620226 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d046a-17e6-4309-bf9e-a65a888f7e49 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:51.620226 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-121d046a-17e6-4309-bf9e-a65a888f7e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:51.620485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413e47c-cfc6-4593-aa0e-7fcc0ce28a85 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:51.703131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa552a-c752-40b0-8903-29cf84a67999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:51.704494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa552a-c752-40b0-8903-29cf84a67999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:51.704935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa552a-c752-40b0-8903-29cf84a67999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:51.705404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa552a-c752-40b0-8903-29cf84a67999 admin admin] PATH_INFO: `/v3/projects/5f79fcbe275b42a1b8cc61f890657afc` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:51.706969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa552a-c752-40b0-8903-29cf84a67999 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f79fcbe275b42a1b8cc61f890657afc)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:51.710202 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa552a-c752-40b0-8903-29cf84a67999 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:51.721893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d046a-17e6-4309-bf9e-a65a888f7e49 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:51.731082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c7092-bf80-4004-be9a-8c8d00da917b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:51.732059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c7092-bf80-4004-be9a-8c8d00da917b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:51.732537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c7092-bf80-4004-be9a-8c8d00da917b admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:51.732766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c7092-bf80-4004-be9a-8c8d00da917b admin admin] PATH_INFO: `/v3/users/7798da08bc8646dd93d4582014c929ae` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:51.734093 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c7092-bf80-4004-be9a-8c8d00da917b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7798da08bc8646dd93d4582014c929ae)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:51.736088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c7092-bf80-4004-be9a-8c8d00da917b admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:51.779101 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cbaa552a-c752-40b0-8903-29cf84a67999 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f79fcbe275b42a1b8cc61f890657afc', 'request_id': 'req-cbaa552a-c752-40b0-8903-29cf84a67999'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:51.791682 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cbaa552a-c752-40b0-8903-29cf84a67999 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f79fcbe275b42a1b8cc61f890657afc', 'request_id': 'req-cbaa552a-c752-40b0-8903-29cf84a67999'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:51.800886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cbaa552a-c752-40b0-8903-29cf84a67999 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f79fcbe275b42a1b8cc61f890657afc', 'request_id': 'req-cbaa552a-c752-40b0-8903-29cf84a67999'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:51.821322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-602c7092-bf80-4004-be9a-8c8d00da917b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7798da08bc8646dd93d4582014c929ae'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:51.821678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cbaa552a-c752-40b0-8903-29cf84a67999 admin admin] The token cache is being invalidate because project 5f79fcbe275b42a1b8cc61f890657afc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:51.822119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cbaa552a-c752-40b0-8903-29cf84a67999 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:51.824777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cbaa552a-c752-40b0-8903-29cf84a67999 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:51.830859 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1038/2101] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:51 2022] DELETE /identity/v3/projects/5f79fcbe275b42a1b8cc61f890657afc => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:51.835688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b2b16-ab99-40c4-9036-913ed1bcccba None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:51.836802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b2b16-ab99-40c4-9036-913ed1bcccba None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:51.837260 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b2b16-ab99-40c4-9036-913ed1bcccba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:51.842190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:51.845099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-602c7092-bf80-4004-be9a-8c8d00da917b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7798da08bc8646dd93d4582014c929ae'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:51.845936 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-602c7092-bf80-4004-be9a-8c8d00da917b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7798da08bc8646dd93d4582014c929ae'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:51.859459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-602c7092-bf80-4004-be9a-8c8d00da917b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7798da08bc8646dd93d4582014c929ae'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:51.862662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-602c7092-bf80-4004-be9a-8c8d00da917b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7798da08bc8646dd93d4582014c929ae'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:51.879595 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1064/2102] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:51 2022] DELETE /identity/v3/users/7798da08bc8646dd93d4582014c929ae => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:31:51.884633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe9af6-616f-482c-933c-05d3cf16f9f2 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:51.885162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe9af6-616f-482c-933c-05d3cf16f9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:51.885552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe9af6-616f-482c-933c-05d3cf16f9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:51.889124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-286b2b16-ab99-40c4-9036-913ed1bcccba None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:51.892307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-286b2b16-ab99-40c4-9036-913ed1bcccba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:51.906029 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ddbe9af6-616f-482c-933c-05d3cf16f9f2 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:51.907183 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ddbe9af6-616f-482c-933c-05d3cf16f9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:51.984963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:51.990116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1065/2103] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:52.000833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe9af6-616f-482c-933c-05d3cf16f9f2 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:52.003147 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ddbe9af6-616f-482c-933c-05d3cf16f9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:52.021241 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1039/2104] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:31:52.023365 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873ea17-2c4d-49f9-baca-e8386ce673a2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:52.024373 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873ea17-2c4d-49f9-baca-e8386ce673a2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:52.025764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873ea17-2c4d-49f9-baca-e8386ce673a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:52.062139 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1873ea17-2c4d-49f9-baca-e8386ce673a2 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:52.063385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1873ea17-2c4d-49f9-baca-e8386ce673a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:52.073303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1040/2105] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:31:52.074702 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873ea17-2c4d-49f9-baca-e8386ce673a2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:52.076270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1873ea17-2c4d-49f9-baca-e8386ce673a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:52.083787 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe9af6-616f-482c-933c-05d3cf16f9f2 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:52.088440 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d70460-e03f-44ed-a747-89e61b2b48e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:52.089793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d70460-e03f-44ed-a747-89e61b2b48e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:52.090240 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873ea17-2c4d-49f9-baca-e8386ce673a2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:52.090730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d70460-e03f-44ed-a747-89e61b2b48e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:52.091207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d70460-e03f-44ed-a747-89e61b2b48e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:52.095409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28189d38-3bee-4185-ba65-6bfdcfc8637e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:52.097258 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28189d38-3bee-4185-ba65-6bfdcfc8637e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:52.097580 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28189d38-3bee-4185-ba65-6bfdcfc8637e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:52.098045 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28189d38-3bee-4185-ba65-6bfdcfc8637e admin admin] PATH_INFO: `/v3/users/68b8d711f6b1463eb4449699d537c4ea` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:52.099411 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28189d38-3bee-4185-ba65-6bfdcfc8637e admin admin] RBAC: Authorizing `identity:delete_user(user_id=68b8d711f6b1463eb4449699d537c4ea)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:52.101991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28189d38-3bee-4185-ba65-6bfdcfc8637e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:52.108301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d70460-e03f-44ed-a747-89e61b2b48e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:52.121527 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d70460-e03f-44ed-a747-89e61b2b48e9 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:52.139557 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1066/2106] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:31:52.140451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d70460-e03f-44ed-a747-89e61b2b48e9 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:52.167648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d70460-e03f-44ed-a747-89e61b2b48e9 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:52.173070 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3df40-e29b-499e-bd57-e9084168ec0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:52.173939 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3df40-e29b-499e-bd57-e9084168ec0f admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:52.174328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3df40-e29b-499e-bd57-e9084168ec0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:52.174703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3df40-e29b-499e-bd57-e9084168ec0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:52.188868 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3df40-e29b-499e-bd57-e9084168ec0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:31:52.190240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3df40-e29b-499e-bd57-e9084168ec0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:52.196261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3df40-e29b-499e-bd57-e9084168ec0f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:52.234547 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1067/2107] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:31:52.236777 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3df40-e29b-499e-bd57-e9084168ec0f admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:52.244948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-28189d38-3bee-4185-ba65-6bfdcfc8637e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68b8d711f6b1463eb4449699d537c4ea'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:52.265788 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3df40-e29b-499e-bd57-e9084168ec0f admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:52.278087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-28189d38-3bee-4185-ba65-6bfdcfc8637e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68b8d711f6b1463eb4449699d537c4ea'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:52.278911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08a24d-8282-4dc9-9f02-bdc85b638559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:52.279585 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08a24d-8282-4dc9-9f02-bdc85b638559 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:52.279862 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08a24d-8282-4dc9-9f02-bdc85b638559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:52.280141 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08a24d-8282-4dc9-9f02-bdc85b638559 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:52.281119 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08a24d-8282-4dc9-9f02-bdc85b638559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:52.282797 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08a24d-8282-4dc9-9f02-bdc85b638559 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:52.284504 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-28189d38-3bee-4185-ba65-6bfdcfc8637e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68b8d711f6b1463eb4449699d537c4ea'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:52.285128 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-28189d38-3bee-4185-ba65-6bfdcfc8637e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68b8d711f6b1463eb4449699d537c4ea'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:52.298894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-28189d38-3bee-4185-ba65-6bfdcfc8637e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68b8d711f6b1463eb4449699d537c4ea'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:52.333192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1041/2108] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:52 2022] DELETE /identity/v3/users/68b8d711f6b1463eb4449699d537c4ea => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:52.336686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66235ff-22fe-4d09-8ea3-8908bee3d4a6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:52.336998 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66235ff-22fe-4d09-8ea3-8908bee3d4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:52.337414 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66235ff-22fe-4d09-8ea3-8908bee3d4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:52.357265 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1068/2109] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:52 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:52.359351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08a24d-8282-4dc9-9f02-bdc85b638559 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:52.359985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f66235ff-22fe-4d09-8ea3-8908bee3d4a6 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:52.361261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f66235ff-22fe-4d09-8ea3-8908bee3d4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:52.364104 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7f08a24d-8282-4dc9-9f02-bdc85b638559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:52.461334 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:52.484808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1042/2110] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:52.488083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08a24d-8282-4dc9-9f02-bdc85b638559 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:52.497524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66235ff-22fe-4d09-8ea3-8908bee3d4a6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:52.498108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42758f-e72d-46ee-9f3a-5320ddfb310a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:52.499053 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f66235ff-22fe-4d09-8ea3-8908bee3d4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:52.499168 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42758f-e72d-46ee-9f3a-5320ddfb310a admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:52.499469 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42758f-e72d-46ee-9f3a-5320ddfb310a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:52.499749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42758f-e72d-46ee-9f3a-5320ddfb310a admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:52.500458 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42758f-e72d-46ee-9f3a-5320ddfb310a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:52.502921 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42758f-e72d-46ee-9f3a-5320ddfb310a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:52.531062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1069/2111] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:52 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:52.532229 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42758f-e72d-46ee-9f3a-5320ddfb310a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:52.545072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42758f-e72d-46ee-9f3a-5320ddfb310a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:52.560335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103456c4-96c7-4207-b014-940c47fc6cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:52.561964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103456c4-96c7-4207-b014-940c47fc6cff admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:52.562356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103456c4-96c7-4207-b014-940c47fc6cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:52.562745 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103456c4-96c7-4207-b014-940c47fc6cff admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:52.563622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103456c4-96c7-4207-b014-940c47fc6cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:52.565389 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103456c4-96c7-4207-b014-940c47fc6cff admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:52.590645 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1070/2112] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:31:52.595691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103456c4-96c7-4207-b014-940c47fc6cff admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:52.598150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66235ff-22fe-4d09-8ea3-8908bee3d4a6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:52.607041 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f3a4b-0457-40f1-b8a5-d0cdf5997c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:52.607863 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f3a4b-0457-40f1-b8a5-d0cdf5997c93 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:52.608152 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f3a4b-0457-40f1-b8a5-d0cdf5997c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:52.608435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f3a4b-0457-40f1-b8a5-d0cdf5997c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:52.614805 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103456c4-96c7-4207-b014-940c47fc6cff admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:52.625610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef76e9d-fd6a-43c2-a976-944572c23a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:52.626340 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef76e9d-fd6a-43c2-a976-944572c23a09 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:52.626628 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef76e9d-fd6a-43c2-a976-944572c23a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:52.626909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef76e9d-fd6a-43c2-a976-944572c23a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:52.628233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef76e9d-fd6a-43c2-a976-944572c23a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:52.631663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef76e9d-fd6a-43c2-a976-944572c23a09 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:52.637232 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f3a4b-0457-40f1-b8a5-d0cdf5997c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:52.638882 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f3a4b-0457-40f1-b8a5-d0cdf5997c93 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:52.657528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1043/2113] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:31:52.658366 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f3a4b-0457-40f1-b8a5-d0cdf5997c93 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:52.662219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e87f3a4b-0457-40f1-b8a5-d0cdf5997c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:52.662353 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1071/2114] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:52.663948 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef76e9d-fd6a-43c2-a976-944572c23a09 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:52.683920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef76e9d-fd6a-43c2-a976-944572c23a09 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:52.687475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f3a4b-0457-40f1-b8a5-d0cdf5997c93 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:52.692869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809c281-f33b-48b0-a5b7-466a07f44663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:52.693040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68833f2e-5eab-45a3-b2d3-88e3981ffca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:52.695501 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809c281-f33b-48b0-a5b7-466a07f44663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:52.695945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809c281-f33b-48b0-a5b7-466a07f44663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:52.696509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809c281-f33b-48b0-a5b7-466a07f44663 admin admin] PATH_INFO: `/v3/projects/6f4961f1bbd8471c99eb298ac785eb5d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:52.698026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7809c281-f33b-48b0-a5b7-466a07f44663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f4961f1bbd8471c99eb298ac785eb5d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:52.700424 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7809c281-f33b-48b0-a5b7-466a07f44663 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:52.703502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68833f2e-5eab-45a3-b2d3-88e3981ffca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:52.703747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:52.704253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68833f2e-5eab-45a3-b2d3-88e3981ffca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:52.705793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68833f2e-5eab-45a3-b2d3-88e3981ffca8 admin admin] PATH_INFO: `/v3/projects/e5b2d3eaf4204ef4b40be51cbb4f7b9b/users/4c67af6122e44ac38fa5f2b9f14bcad5/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:52.727927 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68833f2e-5eab-45a3-b2d3-88e3981ffca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5b2d3eaf4204ef4b40be51cbb4f7b9b, user_id=4c67af6122e44ac38fa5f2b9f14bcad5, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:52.731572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68833f2e-5eab-45a3-b2d3-88e3981ffca8 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:52.760590 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1072/2115] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:52 2022] PUT /identity/v3/projects/e5b2d3eaf4204ef4b40be51cbb4f7b9b/users/4c67af6122e44ac38fa5f2b9f14bcad5/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:52.773891 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e7ecf-9c54-475d-9397-16c826920e33 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:52.774672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e7ecf-9c54-475d-9397-16c826920e33 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:52.774862 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e7ecf-9c54-475d-9397-16c826920e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:52.811113 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-310e7ecf-9c54-475d-9397-16c826920e33 None None] MFA Rules not processed for user `4c67af6122e44ac38fa5f2b9f14bcad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:52.812662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-310e7ecf-9c54-475d-9397-16c826920e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:52.816766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7809c281-f33b-48b0-a5b7-466a07f44663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f4961f1bbd8471c99eb298ac785eb5d', 'request_id': 'req-7809c281-f33b-48b0-a5b7-466a07f44663'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:52.830512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7809c281-f33b-48b0-a5b7-466a07f44663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f4961f1bbd8471c99eb298ac785eb5d', 'request_id': 'req-7809c281-f33b-48b0-a5b7-466a07f44663'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:52.845987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7809c281-f33b-48b0-a5b7-466a07f44663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f4961f1bbd8471c99eb298ac785eb5d', 'request_id': 'req-7809c281-f33b-48b0-a5b7-466a07f44663'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:52.873313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7809c281-f33b-48b0-a5b7-466a07f44663 admin admin] The token cache is being invalidate because project 6f4961f1bbd8471c99eb298ac785eb5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:52.873615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7809c281-f33b-48b0-a5b7-466a07f44663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:52.875586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7809c281-f33b-48b0-a5b7-466a07f44663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:52.877539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1044/2116] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:52 2022] DELETE /identity/v3/projects/6f4961f1bbd8471c99eb298ac785eb5d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:52.883888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed359ef8-75f3-423a-956c-44eed1ed3235 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:52.884005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed359ef8-75f3-423a-956c-44eed1ed3235 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:52.884297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed359ef8-75f3-423a-956c-44eed1ed3235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:52.921381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ed359ef8-75f3-423a-956c-44eed1ed3235 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:52.922980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ed359ef8-75f3-423a-956c-44eed1ed3235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:52.960216 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1073/2117] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:52.972241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e7ecf-9c54-475d-9397-16c826920e33 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:52.974461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-310e7ecf-9c54-475d-9397-16c826920e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:53.063286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1045/2118] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:31:53.071441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f472f-36bd-4e60-b83a-6f4e70fda925 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:53.071958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f472f-36bd-4e60-b83a-6f4e70fda925 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:53.072413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f472f-36bd-4e60-b83a-6f4e70fda925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:53.101931 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e7ecf-9c54-475d-9397-16c826920e33 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:53.109830 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ad052-bd9f-422e-8a8a-06fa95c5efbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:53.112436 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-340f472f-36bd-4e60-b83a-6f4e70fda925 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:53.112963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ad052-bd9f-422e-8a8a-06fa95c5efbc service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:53.113399 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ad052-bd9f-422e-8a8a-06fa95c5efbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:53.113691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ad052-bd9f-422e-8a8a-06fa95c5efbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:53.116876 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-340f472f-36bd-4e60-b83a-6f4e70fda925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:53.136687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ad052-bd9f-422e-8a8a-06fa95c5efbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:53.138190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ad052-bd9f-422e-8a8a-06fa95c5efbc service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:53.143497 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1046/2119] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:53.144352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f472f-36bd-4e60-b83a-6f4e70fda925 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:53.156214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-340f472f-36bd-4e60-b83a-6f4e70fda925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:53.175047 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1074/2120] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:53.176652 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ad052-bd9f-422e-8a8a-06fa95c5efbc service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:53.185625 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f472f-36bd-4e60-b83a-6f4e70fda925 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:53.199548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ad052-bd9f-422e-8a8a-06fa95c5efbc service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:53.205840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eceda3d-39a0-4d2e-9eca-a170702b0e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:53.206248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1f3fb-6c3e-4f66-b037-cd778f4c2190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:53.206987 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1f3fb-6c3e-4f66-b037-cd778f4c2190 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:53.207325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1f3fb-6c3e-4f66-b037-cd778f4c2190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:53.207920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eceda3d-39a0-4d2e-9eca-a170702b0e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:53.208033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1f3fb-6c3e-4f66-b037-cd778f4c2190 admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:53.208491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eceda3d-39a0-4d2e-9eca-a170702b0e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:53.208960 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1f3fb-6c3e-4f66-b037-cd778f4c2190 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:31:53.209760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eceda3d-39a0-4d2e-9eca-a170702b0e98 admin admin] PATH_INFO: `/v3/projects/1e602a9a23df4fbd80447123544cfe58` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:53.210722 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1f3fb-6c3e-4f66-b037-cd778f4c2190 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:53.211897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eceda3d-39a0-4d2e-9eca-a170702b0e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e602a9a23df4fbd80447123544cfe58)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:53.215878 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1f3fb-6c3e-4f66-b037-cd778f4c2190 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:53.218971 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eceda3d-39a0-4d2e-9eca-a170702b0e98 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:53.228706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1075/2121] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:31:53.229606 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1f3fb-6c3e-4f66-b037-cd778f4c2190 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:53.232786 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d2d1f3fb-6c3e-4f66-b037-cd778f4c2190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:53.265927 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1f3fb-6c3e-4f66-b037-cd778f4c2190 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:53.274158 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c3ed5-0a31-49d3-829f-7152eb4ee6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:53.275203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c3ed5-0a31-49d3-829f-7152eb4ee6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:53.275692 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c3ed5-0a31-49d3-829f-7152eb4ee6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:53.276339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c3ed5-0a31-49d3-829f-7152eb4ee6cd admin admin] PATH_INFO: `/v3/users/4c67af6122e44ac38fa5f2b9f14bcad5` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:53.277625 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c3ed5-0a31-49d3-829f-7152eb4ee6cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c67af6122e44ac38fa5f2b9f14bcad5)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:53.280005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c3ed5-0a31-49d3-829f-7152eb4ee6cd admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:53.286181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:53.317098 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8eceda3d-39a0-4d2e-9eca-a170702b0e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e602a9a23df4fbd80447123544cfe58', 'request_id': 'req-8eceda3d-39a0-4d2e-9eca-a170702b0e98'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:53.327416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8eceda3d-39a0-4d2e-9eca-a170702b0e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e602a9a23df4fbd80447123544cfe58', 'request_id': 'req-8eceda3d-39a0-4d2e-9eca-a170702b0e98'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:53.336098 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8eceda3d-39a0-4d2e-9eca-a170702b0e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e602a9a23df4fbd80447123544cfe58', 'request_id': 'req-8eceda3d-39a0-4d2e-9eca-a170702b0e98'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:53.350160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-725c3ed5-0a31-49d3-829f-7152eb4ee6cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c67af6122e44ac38fa5f2b9f14bcad5'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:53.368255 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-725c3ed5-0a31-49d3-829f-7152eb4ee6cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c67af6122e44ac38fa5f2b9f14bcad5'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:53.369389 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-725c3ed5-0a31-49d3-829f-7152eb4ee6cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c67af6122e44ac38fa5f2b9f14bcad5'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:53.369577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8eceda3d-39a0-4d2e-9eca-a170702b0e98 admin admin] The token cache is being invalidate because project 1e602a9a23df4fbd80447123544cfe58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:53.369880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8eceda3d-39a0-4d2e-9eca-a170702b0e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:53.372027 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8eceda3d-39a0-4d2e-9eca-a170702b0e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:53.373496 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1047/2122] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:53 2022] DELETE /identity/v3/projects/1e602a9a23df4fbd80447123544cfe58 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:53.374926 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eceda3d-39a0-4d2e-9eca-a170702b0e98 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:53.379398 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8eceda3d-39a0-4d2e-9eca-a170702b0e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:53.386387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-725c3ed5-0a31-49d3-829f-7152eb4ee6cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c67af6122e44ac38fa5f2b9f14bcad5'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:53.387314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-725c3ed5-0a31-49d3-829f-7152eb4ee6cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c67af6122e44ac38fa5f2b9f14bcad5'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:53.401558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:53.420942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1076/2123] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:31:53 2022] DELETE /identity/v3/users/4c67af6122e44ac38fa5f2b9f14bcad5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:53.423580 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7936d-634c-49f7-9df2-c48146e8117f None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:53.423931 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7936d-634c-49f7-9df2-c48146e8117f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:53.424660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7936d-634c-49f7-9df2-c48146e8117f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:53.455289 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-99c7936d-634c-49f7-9df2-c48146e8117f None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:53.457632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-99c7936d-634c-49f7-9df2-c48146e8117f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:53.502737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eceda3d-39a0-4d2e-9eca-a170702b0e98 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:53.543124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1077/2124] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:53.546644 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e69730-a44e-4c73-bf08-43c991653e64 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:53.546934 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e69730-a44e-4c73-bf08-43c991653e64 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:53.547213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e69730-a44e-4c73-bf08-43c991653e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:53.571340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d3d1e-a8d2-4736-bad6-863a68f29563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:53.572327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d3d1e-a8d2-4736-bad6-863a68f29563 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:53.572840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d3d1e-a8d2-4736-bad6-863a68f29563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:53.573275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d3d1e-a8d2-4736-bad6-863a68f29563 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:53.576467 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d3d1e-a8d2-4736-bad6-863a68f29563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:53.578561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d3d1e-a8d2-4736-bad6-863a68f29563 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:53.578908 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-07e69730-a44e-4c73-bf08-43c991653e64 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:53.580133 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-07e69730-a44e-4c73-bf08-43c991653e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:53.596522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1078/2125] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:53.598037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa0f70-2c35-4740-8667-7d6749d18952 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:53.598324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa0f70-2c35-4740-8667-7d6749d18952 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:53.598595 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa0f70-2c35-4740-8667-7d6749d18952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:53.630453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-94aa0f70-2c35-4740-8667-7d6749d18952 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:53.636338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1048/2126] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:53 2022] POST /identity/v3/projects => generated 385 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:53.636512 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-94aa0f70-2c35-4740-8667-7d6749d18952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:53.636610 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d3d1e-a8d2-4736-bad6-863a68f29563 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:53.638286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-011d3d1e-a8d2-4736-bad6-863a68f29563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:53.710458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d3d1e-a8d2-4736-bad6-863a68f29563 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:53.715624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e26697-a18a-4b38-8c31-04d60d606077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:53.716303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e26697-a18a-4b38-8c31-04d60d606077 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:53.716593 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e26697-a18a-4b38-8c31-04d60d606077 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:53.716877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e26697-a18a-4b38-8c31-04d60d606077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:53.743254 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1079/2127] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:53.744320 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa0f70-2c35-4740-8667-7d6749d18952 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:53.757083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa0f70-2c35-4740-8667-7d6749d18952 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:53.763972 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874151fa-3a18-436d-91ca-f7e7e6a160c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:53.764803 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874151fa-3a18-436d-91ca-f7e7e6a160c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:53.764952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874151fa-3a18-436d-91ca-f7e7e6a160c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:53.765267 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874151fa-3a18-436d-91ca-f7e7e6a160c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:53.766046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874151fa-3a18-436d-91ca-f7e7e6a160c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:53.768190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874151fa-3a18-436d-91ca-f7e7e6a160c5 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:53.787508 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:53.800176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1080/2128] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:53 2022] POST /identity/v3/users => generated 293 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:53.801793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:53.804547 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874151fa-3a18-436d-91ca-f7e7e6a160c5 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:53.820674 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e26697-a18a-4b38-8c31-04d60d606077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:53.821703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874151fa-3a18-436d-91ca-f7e7e6a160c5 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:53.823481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e26697-a18a-4b38-8c31-04d60d606077 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:53.827479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2010a0-e2c1-40f7-bb10-9da39084c314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:53.828348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2010a0-e2c1-40f7-bb10-9da39084c314 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:53.828633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2010a0-e2c1-40f7-bb10-9da39084c314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:53.828910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2010a0-e2c1-40f7-bb10-9da39084c314 admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:53.829711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2010a0-e2c1-40f7-bb10-9da39084c314 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:31:53.830033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2010a0-e2c1-40f7-bb10-9da39084c314 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:53.831636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2010a0-e2c1-40f7-bb10-9da39084c314 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:53.835174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1049/2129] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:53.836020 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e26697-a18a-4b38-8c31-04d60d606077 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:53.846778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e26697-a18a-4b38-8c31-04d60d606077 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:53.848925 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1081/2130] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:31:53.853819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2010a0-e2c1-40f7-bb10-9da39084c314 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:53.857987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff8a66-7837-42ff-bfa8-db6a1e1d482c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:53.858686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff8a66-7837-42ff-bfa8-db6a1e1d482c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:53.858979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff8a66-7837-42ff-bfa8-db6a1e1d482c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:53.859327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff8a66-7837-42ff-bfa8-db6a1e1d482c admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:53.860006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff8a66-7837-42ff-bfa8-db6a1e1d482c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:53.861710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff8a66-7837-42ff-bfa8-db6a1e1d482c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:53.870502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2010a0-e2c1-40f7-bb10-9da39084c314 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:53.876803 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85f1f7-26c5-4d34-a0ec-c73bb48cd613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:53.877429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85f1f7-26c5-4d34-a0ec-c73bb48cd613 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:53.877617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1050/2131] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:53.877903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85f1f7-26c5-4d34-a0ec-c73bb48cd613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:53.878199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85f1f7-26c5-4d34-a0ec-c73bb48cd613 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:53.878832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff8a66-7837-42ff-bfa8-db6a1e1d482c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:53.879008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85f1f7-26c5-4d34-a0ec-c73bb48cd613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:53.881250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85f1f7-26c5-4d34-a0ec-c73bb48cd613 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:53.895788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff8a66-7837-42ff-bfa8-db6a1e1d482c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:53.902598 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12383129-5ff1-4078-9cbe-e4081d5d014f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:53.903837 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12383129-5ff1-4078-9cbe-e4081d5d014f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:53.904261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12383129-5ff1-4078-9cbe-e4081d5d014f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:53.904555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12383129-5ff1-4078-9cbe-e4081d5d014f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:53.905530 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12383129-5ff1-4078-9cbe-e4081d5d014f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:53.907976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12383129-5ff1-4078-9cbe-e4081d5d014f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:53.922765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1082/2132] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:53 2022] POST /identity/v3/projects => generated 399 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:53.924171 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85f1f7-26c5-4d34-a0ec-c73bb48cd613 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:53.930574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1051/2133] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:53.934998 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12383129-5ff1-4078-9cbe-e4081d5d014f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:53.950604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12383129-5ff1-4078-9cbe-e4081d5d014f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:54.014260 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85f1f7-26c5-4d34-a0ec-c73bb48cd613 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:54.020203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604e2de-a5a8-409f-95da-8ec0c064cbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:54.020969 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604e2de-a5a8-409f-95da-8ec0c064cbce admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:54.021286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604e2de-a5a8-409f-95da-8ec0c064cbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:54.022169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604e2de-a5a8-409f-95da-8ec0c064cbce admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:54.022884 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604e2de-a5a8-409f-95da-8ec0c064cbce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:54.025115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604e2de-a5a8-409f-95da-8ec0c064cbce admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:54.057080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1083/2134] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:53 2022] POST /identity/v3/users => generated 300 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:54.060322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604e2de-a5a8-409f-95da-8ec0c064cbce admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:54.085764 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604e2de-a5a8-409f-95da-8ec0c064cbce admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:54.092985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5690d-89f1-45e9-89fa-65c5580aec1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:54.093790 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5690d-89f1-45e9-89fa-65c5580aec1c admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:54.094075 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5690d-89f1-45e9-89fa-65c5580aec1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:54.094352 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5690d-89f1-45e9-89fa-65c5580aec1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:54.095699 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5690d-89f1-45e9-89fa-65c5580aec1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:54.098130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5690d-89f1-45e9-89fa-65c5580aec1c admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:54.114911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1084/2135] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:54.119199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5690d-89f1-45e9-89fa-65c5580aec1c admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:54.127292 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84aced-5ac9-4428-8ccd-fc71aa61a772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:54.128687 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84aced-5ac9-4428-8ccd-fc71aa61a772 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:54.129279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84aced-5ac9-4428-8ccd-fc71aa61a772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:54.129713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84aced-5ac9-4428-8ccd-fc71aa61a772 admin admin] PATH_INFO: `/v3/projects/a96895fbf8bd47d5a319ea1a20bef3be/users/1b7cfb7733494aeeba01cfc2d29a334c/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:54.136063 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5690d-89f1-45e9-89fa-65c5580aec1c admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:54.141646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16429741-979d-45a6-b195-33fdf103a254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:54.142998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16429741-979d-45a6-b195-33fdf103a254 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:54.143303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16429741-979d-45a6-b195-33fdf103a254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:54.143804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16429741-979d-45a6-b195-33fdf103a254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:54.145175 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16429741-979d-45a6-b195-33fdf103a254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:54.146932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16429741-979d-45a6-b195-33fdf103a254 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:54.156575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84aced-5ac9-4428-8ccd-fc71aa61a772 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a96895fbf8bd47d5a319ea1a20bef3be, user_id=1b7cfb7733494aeeba01cfc2d29a334c, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:54.161548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84aced-5ac9-4428-8ccd-fc71aa61a772 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:54.178838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1085/2136] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:54.181065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1052/2137] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:53 2022] PUT /identity/v3/projects/a96895fbf8bd47d5a319ea1a20bef3be/users/1b7cfb7733494aeeba01cfc2d29a334c/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:54.192450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16429741-979d-45a6-b195-33fdf103a254 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:54.204257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3121c-431d-4592-805c-9f829dcb73d0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:54.206013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3121c-431d-4592-805c-9f829dcb73d0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:54.206463 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3121c-431d-4592-805c-9f829dcb73d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:54.207486 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16429741-979d-45a6-b195-33fdf103a254 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:54.306812 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4ea3121c-431d-4592-805c-9f829dcb73d0 None None] MFA Rules not processed for user `1b7cfb7733494aeeba01cfc2d29a334c`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:54.308799 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4ea3121c-431d-4592-805c-9f829dcb73d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:54.377700 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a200e3-4be7-467f-9cb7-ba90cbef1878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:54.378525 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a200e3-4be7-467f-9cb7-ba90cbef1878 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:54.378827 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a200e3-4be7-467f-9cb7-ba90cbef1878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:54.379399 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a200e3-4be7-467f-9cb7-ba90cbef1878 admin admin] PATH_INFO: `/v3/projects/3253a3c2fb474da781ef0bff7b4a0b75/users/48fd6a1f1d61494592ae640d4fee2df1/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:54.402140 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a200e3-4be7-467f-9cb7-ba90cbef1878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3253a3c2fb474da781ef0bff7b4a0b75, user_id=48fd6a1f1d61494592ae640d4fee2df1, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:54.407678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a200e3-4be7-467f-9cb7-ba90cbef1878 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:54.432427 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1086/2138] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:54 2022] PUT /identity/v3/projects/3253a3c2fb474da781ef0bff7b4a0b75/users/48fd6a1f1d61494592ae640d4fee2df1/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:54.433889 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a200e3-4be7-467f-9cb7-ba90cbef1878 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:54.435575 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d0a200e3-4be7-467f-9cb7-ba90cbef1878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:54.461671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1053/2139] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:54.462574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3121c-431d-4592-805c-9f829dcb73d0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:54.463917 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4ea3121c-431d-4592-805c-9f829dcb73d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:54.472694 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:54.494079 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3121c-431d-4592-805c-9f829dcb73d0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:54.537799 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a200e3-4be7-467f-9cb7-ba90cbef1878 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:54.546623 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb9274-a6f1-43de-86a0-d8fd20b93085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:54.547471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb9274-a6f1-43de-86a0-d8fd20b93085 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:54.549305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb9274-a6f1-43de-86a0-d8fd20b93085 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:54.549305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb9274-a6f1-43de-86a0-d8fd20b93085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:54.552496 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ddeb9274-a6f1-43de-86a0-d8fd20b93085 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:54.562037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:54.587488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63ace6-b316-468e-b371-5fc99dc96063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:54.588427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63ace6-b316-468e-b371-5fc99dc96063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:54.588717 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63ace6-b316-468e-b371-5fc99dc96063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:54.589223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63ace6-b316-468e-b371-5fc99dc96063 admin admin] PATH_INFO: `/v3/projects/e5b2d3eaf4204ef4b40be51cbb4f7b9b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:54.590457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63ace6-b316-468e-b371-5fc99dc96063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5b2d3eaf4204ef4b40be51cbb4f7b9b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:54.592398 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63ace6-b316-468e-b371-5fc99dc96063 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:54.676827 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb9274-a6f1-43de-86a0-d8fd20b93085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:54.679287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb9274-a6f1-43de-86a0-d8fd20b93085 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:54.686655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b63ace6-b316-468e-b371-5fc99dc96063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5b2d3eaf4204ef4b40be51cbb4f7b9b', 'request_id': 'req-4b63ace6-b316-468e-b371-5fc99dc96063'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:54.698485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b63ace6-b316-468e-b371-5fc99dc96063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5b2d3eaf4204ef4b40be51cbb4f7b9b', 'request_id': 'req-4b63ace6-b316-468e-b371-5fc99dc96063'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:54.701776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1087/2140] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:54.702440 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ad4df-0f66-43b4-a6a7-1d9dc75ff340 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:54.702779 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ad4df-0f66-43b4-a6a7-1d9dc75ff340 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:54.703021 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ad4df-0f66-43b4-a6a7-1d9dc75ff340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:54.727076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b63ace6-b316-468e-b371-5fc99dc96063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5b2d3eaf4204ef4b40be51cbb4f7b9b', 'request_id': 'req-4b63ace6-b316-468e-b371-5fc99dc96063'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:54.751241 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b63ace6-b316-468e-b371-5fc99dc96063 admin admin] The token cache is being invalidate because project e5b2d3eaf4204ef4b40be51cbb4f7b9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:31:54.751363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b63ace6-b316-468e-b371-5fc99dc96063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:54.752946 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b63ace6-b316-468e-b371-5fc99dc96063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:31:54.755064 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1054/2141] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:31:54 2022] DELETE /identity/v3/projects/e5b2d3eaf4204ef4b40be51cbb4f7b9b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:54.759144 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63ace6-b316-468e-b371-5fc99dc96063 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:54.759255 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-3d2ad4df-0f66-43b4-a6a7-1d9dc75ff340 None None] MFA Rules not processed for user `48fd6a1f1d61494592ae640d4fee2df1`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:54.760428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4b63ace6-b316-468e-b371-5fc99dc96063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:54.763200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3d2ad4df-0f66-43b4-a6a7-1d9dc75ff340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:54.886450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63ace6-b316-468e-b371-5fc99dc96063 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:54.891071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5fc27-f3a2-4de2-bc73-67326ba33f66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:54.891922 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5fc27-f3a2-4de2-bc73-67326ba33f66 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:54.892282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5fc27-f3a2-4de2-bc73-67326ba33f66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:54.892633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5fc27-f3a2-4de2-bc73-67326ba33f66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:54.894783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-23c5fc27-f3a2-4de2-bc73-67326ba33f66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:54.922161 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1088/2142] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:54.923773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b2f6b-2f6b-4a8f-aa60-ee2e7bbcf0f9 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:54.924351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b2f6b-2f6b-4a8f-aa60-ee2e7bbcf0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:54.924522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b2f6b-2f6b-4a8f-aa60-ee2e7bbcf0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:54.976989 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-f11b2f6b-2f6b-4a8f-aa60-ee2e7bbcf0f9 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:54.981726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f11b2f6b-2f6b-4a8f-aa60-ee2e7bbcf0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:55.005501 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5fc27-f3a2-4de2-bc73-67326ba33f66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:55.008715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5fc27-f3a2-4de2-bc73-67326ba33f66 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:55.023862 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1055/2143] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:55.039106 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5fc27-f3a2-4de2-bc73-67326ba33f66 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:55.048573 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-23c5fc27-f3a2-4de2-bc73-67326ba33f66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:55.078139 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:55.084825 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5fc27-f3a2-4de2-bc73-67326ba33f66 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:55.094123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa92fe9-2fc5-46ca-85d5-c9723b0078b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:55.094807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa92fe9-2fc5-46ca-85d5-c9723b0078b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:55.095092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa92fe9-2fc5-46ca-85d5-c9723b0078b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:55.095369 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa92fe9-2fc5-46ca-85d5-c9723b0078b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:55.095675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1089/2144] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:55.105189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c97a9-2932-43e1-9dda-d2feeaf3e6b5 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:55.105712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c97a9-2932-43e1-9dda-d2feeaf3e6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:55.106100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c97a9-2932-43e1-9dda-d2feeaf3e6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:55.118356 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa92fe9-2fc5-46ca-85d5-c9723b0078b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:55.121113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa92fe9-2fc5-46ca-85d5-c9723b0078b9 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:55.130021 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-283c97a9-2932-43e1-9dda-d2feeaf3e6b5 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:55.132372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-283c97a9-2932-43e1-9dda-d2feeaf3e6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:55.157206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1090/2145] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:55.161124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c97a9-2932-43e1-9dda-d2feeaf3e6b5 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:55.171293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1056/2146] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:31:55.183629 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c97a9-2932-43e1-9dda-d2feeaf3e6b5 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:55.199424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f306512-656b-43d3-ae87-ae995b8bb6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:55.200124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f306512-656b-43d3-ae87-ae995b8bb6cb admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:55.200413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f306512-656b-43d3-ae87-ae995b8bb6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:55.200693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f306512-656b-43d3-ae87-ae995b8bb6cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:55.202931 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f306512-656b-43d3-ae87-ae995b8bb6cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:31:55.202931 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f306512-656b-43d3-ae87-ae995b8bb6cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:55.210907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f306512-656b-43d3-ae87-ae995b8bb6cb admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:55.239179 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa92fe9-2fc5-46ca-85d5-c9723b0078b9 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:55.247210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1091/2147] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:31:55.259936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa92fe9-2fc5-46ca-85d5-c9723b0078b9 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:55.271074 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d0d00-3239-4fe0-8251-27585dccf3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:55.271735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d0d00-3239-4fe0-8251-27585dccf3c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:55.272379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d0d00-3239-4fe0-8251-27585dccf3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:55.272693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d0d00-3239-4fe0-8251-27585dccf3c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:55.273435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d0d00-3239-4fe0-8251-27585dccf3c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:55.275638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d0d00-3239-4fe0-8251-27585dccf3c7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:55.294265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:55.324905 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1057/2148] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:55 2022] POST /identity/v3/projects => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:55.330274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f306512-656b-43d3-ae87-ae995b8bb6cb admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:55.352151 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f306512-656b-43d3-ae87-ae995b8bb6cb admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:55.463166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dad49-b5a8-48e7-8a43-3d83385daddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:55.464075 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dad49-b5a8-48e7-8a43-3d83385daddc admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:55.464707 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dad49-b5a8-48e7-8a43-3d83385daddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:55.464707 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dad49-b5a8-48e7-8a43-3d83385daddc admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:55.465675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dad49-b5a8-48e7-8a43-3d83385daddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:55.470128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dad49-b5a8-48e7-8a43-3d83385daddc admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:55.506795 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1092/2149] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:55 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:55.511259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d0d00-3239-4fe0-8251-27585dccf3c7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:55.530973 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d0d00-3239-4fe0-8251-27585dccf3c7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:55.541116 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313739f-4834-43cb-a0ec-99b83bbe5541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:55.541859 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313739f-4834-43cb-a0ec-99b83bbe5541 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:55.542141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313739f-4834-43cb-a0ec-99b83bbe5541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:55.542421 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313739f-4834-43cb-a0ec-99b83bbe5541 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:55.543166 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b313739f-4834-43cb-a0ec-99b83bbe5541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:55.545076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b313739f-4834-43cb-a0ec-99b83bbe5541 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:55.606481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1058/2150] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:55.610453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dad49-b5a8-48e7-8a43-3d83385daddc admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:55.628680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dad49-b5a8-48e7-8a43-3d83385daddc admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:55.646169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f175836-08a2-489e-a127-1ae4846e71da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:55.647882 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f175836-08a2-489e-a127-1ae4846e71da admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:55.648754 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f175836-08a2-489e-a127-1ae4846e71da admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:55.649560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f175836-08a2-489e-a127-1ae4846e71da admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:55.650703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f175836-08a2-489e-a127-1ae4846e71da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:55.652794 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f175836-08a2-489e-a127-1ae4846e71da admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:55.667786 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1093/2151] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:55.676172 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313739f-4834-43cb-a0ec-99b83bbe5541 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:55.693464 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313739f-4834-43cb-a0ec-99b83bbe5541 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:55.712843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193f41b-0079-4eff-977b-97f4f1dd745a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:55.714066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193f41b-0079-4eff-977b-97f4f1dd745a admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:55.714794 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193f41b-0079-4eff-977b-97f4f1dd745a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:55.715773 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193f41b-0079-4eff-977b-97f4f1dd745a admin admin] PATH_INFO: `/v3/projects/8d4c2a62dfbf4c7e99fea8837ad85749/users/fd3dc5dde61d411d82a962b40feecb5b/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:55.752580 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193f41b-0079-4eff-977b-97f4f1dd745a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4c2a62dfbf4c7e99fea8837ad85749, user_id=fd3dc5dde61d411d82a962b40feecb5b, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:55.758334 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193f41b-0079-4eff-977b-97f4f1dd745a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:55.808286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1059/2152] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:55 2022] PUT /identity/v3/projects/8d4c2a62dfbf4c7e99fea8837ad85749/users/fd3dc5dde61d411d82a962b40feecb5b/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:55.817718 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69d197-f010-440c-b1b7-2ce7467495ae None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:55.818003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69d197-f010-440c-b1b7-2ce7467495ae None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:55.819056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69d197-f010-440c-b1b7-2ce7467495ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:55.868797 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-8b69d197-f010-440c-b1b7-2ce7467495ae None None] MFA Rules not processed for user `fd3dc5dde61d411d82a962b40feecb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:55.871044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-8b69d197-f010-440c-b1b7-2ce7467495ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:55.981079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1094/2153] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:55.981969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193f41b-0079-4eff-977b-97f4f1dd745a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:56.080093 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193f41b-0079-4eff-977b-97f4f1dd745a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:56.086150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be808604-c5a8-49c1-a567-cfcf25c26088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:56.086889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be808604-c5a8-49c1-a567-cfcf25c26088 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:56.087180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be808604-c5a8-49c1-a567-cfcf25c26088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:56.087470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be808604-c5a8-49c1-a567-cfcf25c26088 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:56.088180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be808604-c5a8-49c1-a567-cfcf25c26088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:56.090199 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be808604-c5a8-49c1-a567-cfcf25c26088 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:56.141185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1060/2154] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:55 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:56.144955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69d197-f010-440c-b1b7-2ce7467495ae None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:56.157385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69d197-f010-440c-b1b7-2ce7467495ae None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:56.212594 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:56.230154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e9f45-9f39-42ce-8695-2581d3d06185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:56.230832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e9f45-9f39-42ce-8695-2581d3d06185 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:56.231114 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e9f45-9f39-42ce-8695-2581d3d06185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:56.231393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e9f45-9f39-42ce-8695-2581d3d06185 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:56.232097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e9f45-9f39-42ce-8695-2581d3d06185 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:56.234083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e9f45-9f39-42ce-8695-2581d3d06185 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:56.254480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1095/2155] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:56 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:56.259313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be808604-c5a8-49c1-a567-cfcf25c26088 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:56.269611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be808604-c5a8-49c1-a567-cfcf25c26088 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:56.274986 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a4348-c757-47fd-b125-54bf9beee982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:56.275667 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a4348-c757-47fd-b125-54bf9beee982 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:56.275952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a4348-c757-47fd-b125-54bf9beee982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:56.276230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a4348-c757-47fd-b125-54bf9beee982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:56.276958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a4348-c757-47fd-b125-54bf9beee982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:56.278583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a4348-c757-47fd-b125-54bf9beee982 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:56.289273 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1061/2156] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:31:56.293006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e9f45-9f39-42ce-8695-2581d3d06185 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:56.303605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e9f45-9f39-42ce-8695-2581d3d06185 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:56.308892 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6aacb-098d-4f7c-9789-22e997c5d2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:56.309661 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6aacb-098d-4f7c-9789-22e997c5d2be admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:56.310007 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6aacb-098d-4f7c-9789-22e997c5d2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:56.310368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6aacb-098d-4f7c-9789-22e997c5d2be admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:56.311173 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6aacb-098d-4f7c-9789-22e997c5d2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:56.312828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6aacb-098d-4f7c-9789-22e997c5d2be admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:56.323903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1096/2157] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:56.327884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a4348-c757-47fd-b125-54bf9beee982 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:56.337842 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a4348-c757-47fd-b125-54bf9beee982 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:56.343780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b27105-456b-4f87-9e5c-4fdf70e22c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:56.344689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b27105-456b-4f87-9e5c-4fdf70e22c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:56.344967 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b27105-456b-4f87-9e5c-4fdf70e22c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:56.345289 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b27105-456b-4f87-9e5c-4fdf70e22c4a admin admin] PATH_INFO: `/v3/projects/a03de89f20d54a57b8c02c45ded8d3b8/users/3a7cfaa194804f1e94ac12971f935fc4/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:56.360451 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b27105-456b-4f87-9e5c-4fdf70e22c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a03de89f20d54a57b8c02c45ded8d3b8, user_id=3a7cfaa194804f1e94ac12971f935fc4, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:56.363190 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b27105-456b-4f87-9e5c-4fdf70e22c4a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:56.376630 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1062/2158] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:56 2022] PUT /identity/v3/projects/a03de89f20d54a57b8c02c45ded8d3b8/users/3a7cfaa194804f1e94ac12971f935fc4/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:56.444908 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb42dc-8554-433f-8fab-21ba79f61a4d None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:56.445224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb42dc-8554-433f-8fab-21ba79f61a4d None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:56.445504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb42dc-8554-433f-8fab-21ba79f61a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:56.471370 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-32fb42dc-8554-433f-8fab-21ba79f61a4d None None] MFA Rules not processed for user `3a7cfaa194804f1e94ac12971f935fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:56.472563 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-32fb42dc-8554-433f-8fab-21ba79f61a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:56.535081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1097/2159] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:31:56.539648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b27105-456b-4f87-9e5c-4fdf70e22c4a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:56.600503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b27105-456b-4f87-9e5c-4fdf70e22c4a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:56.605212 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10558ef-360a-419d-be9b-2e24ad238d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:56.605867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10558ef-360a-419d-be9b-2e24ad238d70 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:56.606157 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10558ef-360a-419d-be9b-2e24ad238d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:56.606439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10558ef-360a-419d-be9b-2e24ad238d70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:56.607143 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10558ef-360a-419d-be9b-2e24ad238d70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:56.609063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10558ef-360a-419d-be9b-2e24ad238d70 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:56.619293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:56.636314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1063/2160] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:31:56 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:31:56.640327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb42dc-8554-433f-8fab-21ba79f61a4d None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:56.651386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb42dc-8554-433f-8fab-21ba79f61a4d None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:56.707741 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d867b-f15b-483a-8776-bfdabef6862f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:56.708411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d867b-f15b-483a-8776-bfdabef6862f admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:56.708723 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d867b-f15b-483a-8776-bfdabef6862f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:56.709018 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d867b-f15b-483a-8776-bfdabef6862f admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:56.709723 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d867b-f15b-483a-8776-bfdabef6862f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:56.711656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d867b-f15b-483a-8776-bfdabef6862f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:56.735550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1098/2161] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:31:56 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:31:56.739480 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10558ef-360a-419d-be9b-2e24ad238d70 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:56.750014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10558ef-360a-419d-be9b-2e24ad238d70 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:56.755027 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095becb-b455-4b3d-a119-80390bfaa46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:56.756194 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095becb-b455-4b3d-a119-80390bfaa46e admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:56.756484 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095becb-b455-4b3d-a119-80390bfaa46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:56.756914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095becb-b455-4b3d-a119-80390bfaa46e admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:56.757697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095becb-b455-4b3d-a119-80390bfaa46e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:56.759318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095becb-b455-4b3d-a119-80390bfaa46e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:56.770798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1064/2162] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:56.778263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d867b-f15b-483a-8776-bfdabef6862f admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:56.789543 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d867b-f15b-483a-8776-bfdabef6862f admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:56.794323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb80933-39ff-44d5-9683-fcd23a72a9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:56.795247 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb80933-39ff-44d5-9683-fcd23a72a9eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:56.795536 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb80933-39ff-44d5-9683-fcd23a72a9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:56.795815 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb80933-39ff-44d5-9683-fcd23a72a9eb admin admin] PATH_INFO: `/v3/projects/caf9021f4fde43c68357c9a595cc4c5f/users/ef2228b364b14c8693df3bbd2fada5b4/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:56.810697 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb80933-39ff-44d5-9683-fcd23a72a9eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=caf9021f4fde43c68357c9a595cc4c5f, user_id=ef2228b364b14c8693df3bbd2fada5b4, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:56.813395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb80933-39ff-44d5-9683-fcd23a72a9eb admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:56.826680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1099/2163] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:56 2022] PUT /identity/v3/projects/caf9021f4fde43c68357c9a595cc4c5f/users/ef2228b364b14c8693df3bbd2fada5b4/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:56.830485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095becb-b455-4b3d-a119-80390bfaa46e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:56.841311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095becb-b455-4b3d-a119-80390bfaa46e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:56.899167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16b1c1-2815-4ea3-b3d1-e0a4f53341f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:56.899841 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16b1c1-2815-4ea3-b3d1-e0a4f53341f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:56.900121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16b1c1-2815-4ea3-b3d1-e0a4f53341f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:56.900399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16b1c1-2815-4ea3-b3d1-e0a4f53341f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:56.901122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16b1c1-2815-4ea3-b3d1-e0a4f53341f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:56.902736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16b1c1-2815-4ea3-b3d1-e0a4f53341f5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:56.913119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1065/2164] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:31:56.917621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb80933-39ff-44d5-9683-fcd23a72a9eb admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:56.927898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb80933-39ff-44d5-9683-fcd23a72a9eb admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:56.932835 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded98c4-ffc2-4592-9bcf-2a4b0499462a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:56.933775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded98c4-ffc2-4592-9bcf-2a4b0499462a admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:56.934062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded98c4-ffc2-4592-9bcf-2a4b0499462a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:56.934344 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded98c4-ffc2-4592-9bcf-2a4b0499462a admin admin] PATH_INFO: `/v3/projects/caf9021f4fde43c68357c9a595cc4c5f/users/ef2228b364b14c8693df3bbd2fada5b4/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:56.936344 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded98c4-ffc2-4592-9bcf-2a4b0499462a admin admin] RBAC: Authorizing `identity:create_grant(project_id=caf9021f4fde43c68357c9a595cc4c5f, user_id=ef2228b364b14c8693df3bbd2fada5b4, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:56.939127 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded98c4-ffc2-4592-9bcf-2a4b0499462a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:56.950847 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1100/2165] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:31:56 2022] PUT /identity/v3/projects/caf9021f4fde43c68357c9a595cc4c5f/users/ef2228b364b14c8693df3bbd2fada5b4/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:31:56.961818 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59384e9-068f-46a3-a958-aeb35a337163 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:56.962127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59384e9-068f-46a3-a958-aeb35a337163 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:56.962393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59384e9-068f-46a3-a958-aeb35a337163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:56.989416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c59384e9-068f-46a3-a958-aeb35a337163 None None] MFA Rules not processed for user `ef2228b364b14c8693df3bbd2fada5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:31:56.990899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c59384e9-068f-46a3-a958-aeb35a337163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:31:57.063349 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1066/2166] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:31:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:31:57.075091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded98c4-ffc2-4592-9bcf-2a4b0499462a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:57.080685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:57.131885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded98c4-ffc2-4592-9bcf-2a4b0499462a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:57.135565 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8019b6-2e07-4493-96bc-309775b5826b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:57.136206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8019b6-2e07-4493-96bc-309775b5826b service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:57.136490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8019b6-2e07-4493-96bc-309775b5826b service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:57.136767 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8019b6-2e07-4493-96bc-309775b5826b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:57.195368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8019b6-2e07-4493-96bc-309775b5826b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:57.196846 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8019b6-2e07-4493-96bc-309775b5826b service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:57.206237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1101/2167] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:31:59.749665 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59384e9-068f-46a3-a958-aeb35a337163 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:31:59.760035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59384e9-068f-46a3-a958-aeb35a337163 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:31:59.763679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee85e1f-175f-4729-a76d-a39c1e9f1417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:31:59.764349 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee85e1f-175f-4729-a76d-a39c1e9f1417 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:31:59.764638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee85e1f-175f-4729-a76d-a39c1e9f1417 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:31:59.764916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee85e1f-175f-4729-a76d-a39c1e9f1417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:31:59.884130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee85e1f-175f-4729-a76d-a39c1e9f1417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:31:59.889236 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee85e1f-175f-4729-a76d-a39c1e9f1417 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:31:59.895146 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:31:59.916337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1067/2168] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:31:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:32:00.873938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8019b6-2e07-4493-96bc-309775b5826b service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:00.885634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8019b6-2e07-4493-96bc-309775b5826b service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:00.891963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77919c8-5d46-45a0-b182-4aceb727c370 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:00.893028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77919c8-5d46-45a0-b182-4aceb727c370 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:00.893429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77919c8-5d46-45a0-b182-4aceb727c370 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:00.893933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77919c8-5d46-45a0-b182-4aceb727c370 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:00.909117 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77919c8-5d46-45a0-b182-4aceb727c370 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:00.910996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77919c8-5d46-45a0-b182-4aceb727c370 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:00.922944 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1102/2169] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:32:10.616745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee85e1f-175f-4729-a76d-a39c1e9f1417 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:10.618965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8ee85e1f-175f-4729-a76d-a39c1e9f1417 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:10.773046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee85e1f-175f-4729-a76d-a39c1e9f1417 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:10.778563 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954641d-ac62-4354-9f06-6d60083dc7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:10.779536 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954641d-ac62-4354-9f06-6d60083dc7df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:10.779820 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954641d-ac62-4354-9f06-6d60083dc7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:10.780180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954641d-ac62-4354-9f06-6d60083dc7df admin admin] PATH_INFO: `/v3/users/1b7cfb7733494aeeba01cfc2d29a334c` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:10.781601 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954641d-ac62-4354-9f06-6d60083dc7df admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b7cfb7733494aeeba01cfc2d29a334c)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:10.783738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954641d-ac62-4354-9f06-6d60083dc7df admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:10.844861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9954641d-ac62-4354-9f06-6d60083dc7df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b7cfb7733494aeeba01cfc2d29a334c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:10.883096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9954641d-ac62-4354-9f06-6d60083dc7df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b7cfb7733494aeeba01cfc2d29a334c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:10.884129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9954641d-ac62-4354-9f06-6d60083dc7df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b7cfb7733494aeeba01cfc2d29a334c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:10.884736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9954641d-ac62-4354-9f06-6d60083dc7df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b7cfb7733494aeeba01cfc2d29a334c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:10.892881 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9954641d-ac62-4354-9f06-6d60083dc7df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b7cfb7733494aeeba01cfc2d29a334c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:10.909962 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1068/2170] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:32:10 2022] DELETE /identity/v3/users/1b7cfb7733494aeeba01cfc2d29a334c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:10.915047 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c502ab-a64b-442e-a64e-8c32bc205185 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:10.915865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c502ab-a64b-442e-a64e-8c32bc205185 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:10.916315 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c502ab-a64b-442e-a64e-8c32bc205185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:10.918217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=52 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:10.944951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-06c502ab-a64b-442e-a64e-8c32bc205185 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:10.946271 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-06c502ab-a64b-442e-a64e-8c32bc205185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:11.013297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1103/2171] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:11.025404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954641d-ac62-4354-9f06-6d60083dc7df admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:11.028620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9954641d-ac62-4354-9f06-6d60083dc7df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:11.096080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954641d-ac62-4354-9f06-6d60083dc7df admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:11.100975 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf622ad-9870-4912-9061-b94c6465de76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:11.101674 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf622ad-9870-4912-9061-b94c6465de76 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:11.101956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf622ad-9870-4912-9061-b94c6465de76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:11.102235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf622ad-9870-4912-9061-b94c6465de76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:11.112123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf622ad-9870-4912-9061-b94c6465de76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:11.113313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf622ad-9870-4912-9061-b94c6465de76 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:11.126603 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1069/2172] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:32:11.464512 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c502ab-a64b-442e-a64e-8c32bc205185 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:11.466147 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-06c502ab-a64b-442e-a64e-8c32bc205185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:11.485658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c502ab-a64b-442e-a64e-8c32bc205185 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:11.498632 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d4b4f-5f85-43c0-be6f-3ad2a4411005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:11.499515 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d4b4f-5f85-43c0-be6f-3ad2a4411005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:11.499798 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d4b4f-5f85-43c0-be6f-3ad2a4411005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:11.500076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d4b4f-5f85-43c0-be6f-3ad2a4411005 admin admin] PATH_INFO: `/v3/projects/a96895fbf8bd47d5a319ea1a20bef3be` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:11.501493 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d4b4f-5f85-43c0-be6f-3ad2a4411005 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a96895fbf8bd47d5a319ea1a20bef3be)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:11.503408 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d4b4f-5f85-43c0-be6f-3ad2a4411005 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:11.560899 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-869d4b4f-5f85-43c0-be6f-3ad2a4411005 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a96895fbf8bd47d5a319ea1a20bef3be', 'request_id': 'req-869d4b4f-5f85-43c0-be6f-3ad2a4411005'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:11.569409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-869d4b4f-5f85-43c0-be6f-3ad2a4411005 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a96895fbf8bd47d5a319ea1a20bef3be', 'request_id': 'req-869d4b4f-5f85-43c0-be6f-3ad2a4411005'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:11.572495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf622ad-9870-4912-9061-b94c6465de76 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:11.574138 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ccf622ad-9870-4912-9061-b94c6465de76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:11.576657 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-869d4b4f-5f85-43c0-be6f-3ad2a4411005 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a96895fbf8bd47d5a319ea1a20bef3be', 'request_id': 'req-869d4b4f-5f85-43c0-be6f-3ad2a4411005'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:11.589555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf622ad-9870-4912-9061-b94c6465de76 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:11.592663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-869d4b4f-5f85-43c0-be6f-3ad2a4411005 admin admin] The token cache is being invalidate because project a96895fbf8bd47d5a319ea1a20bef3be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:32:11.592904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-869d4b4f-5f85-43c0-be6f-3ad2a4411005 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:11.593984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-869d4b4f-5f85-43c0-be6f-3ad2a4411005 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:11.595459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1104/2173] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:32:11 2022] DELETE /identity/v3/projects/a96895fbf8bd47d5a319ea1a20bef3be => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:32:11.602019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46047d5-9f28-4167-9bd7-91350960da0a None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:11.602356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46047d5-9f28-4167-9bd7-91350960da0a None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:11.602582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46047d5-9f28-4167-9bd7-91350960da0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:11.613084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:11.631647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-f46047d5-9f28-4167-9bd7-91350960da0a None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:11.631647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f46047d5-9f28-4167-9bd7-91350960da0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:11.660292 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b068f78-dbdf-4db3-9ecd-b6a6fdd3973d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:11.661342 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b068f78-dbdf-4db3-9ecd-b6a6fdd3973d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:11.661838 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b068f78-dbdf-4db3-9ecd-b6a6fdd3973d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:11.662321 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b068f78-dbdf-4db3-9ecd-b6a6fdd3973d admin admin] PATH_INFO: `/v3/users/48fd6a1f1d61494592ae640d4fee2df1` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:11.663651 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b068f78-dbdf-4db3-9ecd-b6a6fdd3973d admin admin] RBAC: Authorizing `identity:delete_user(user_id=48fd6a1f1d61494592ae640d4fee2df1)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:11.665649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b068f78-dbdf-4db3-9ecd-b6a6fdd3973d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:11.698844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1105/2174] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:11.706855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081df772-34fb-4ede-ba77-58f8b56c27c7 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:11.707087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081df772-34fb-4ede-ba77-58f8b56c27c7 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:11.707387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081df772-34fb-4ede-ba77-58f8b56c27c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:11.717264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0b068f78-dbdf-4db3-9ecd-b6a6fdd3973d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48fd6a1f1d61494592ae640d4fee2df1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:11.725777 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-081df772-34fb-4ede-ba77-58f8b56c27c7 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:11.727046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-081df772-34fb-4ede-ba77-58f8b56c27c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:11.730244 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0b068f78-dbdf-4db3-9ecd-b6a6fdd3973d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48fd6a1f1d61494592ae640d4fee2df1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:11.730916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0b068f78-dbdf-4db3-9ecd-b6a6fdd3973d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48fd6a1f1d61494592ae640d4fee2df1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:11.731546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0b068f78-dbdf-4db3-9ecd-b6a6fdd3973d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48fd6a1f1d61494592ae640d4fee2df1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:11.735315 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1106/2175] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:32:11.738648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0b068f78-dbdf-4db3-9ecd-b6a6fdd3973d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48fd6a1f1d61494592ae640d4fee2df1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:11.739439 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081df772-34fb-4ede-ba77-58f8b56c27c7 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:11.748890 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081df772-34fb-4ede-ba77-58f8b56c27c7 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:11.752309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1070/2176] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:32:11 2022] DELETE /identity/v3/users/48fd6a1f1d61494592ae640d4fee2df1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:32:11.754331 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfbe36-7614-44e2-883d-38025c39a00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:11.755690 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfbe36-7614-44e2-883d-38025c39a00c admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:11.756077 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfbe36-7614-44e2-883d-38025c39a00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:11.756340 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfbe36-7614-44e2-883d-38025c39a00c admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:11.757071 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfbe36-7614-44e2-883d-38025c39a00c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:32:11.757264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d7e74-ecf7-4829-9544-078721e25b56 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:11.757414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfbe36-7614-44e2-883d-38025c39a00c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:11.757561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d7e74-ecf7-4829-9544-078721e25b56 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:11.757847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d7e74-ecf7-4829-9544-078721e25b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:11.759121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfbe36-7614-44e2-883d-38025c39a00c admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:11.775355 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1107/2177] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:32:11.776154 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f22d7e74-ecf7-4829-9544-078721e25b56 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:11.776339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfbe36-7614-44e2-883d-38025c39a00c admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:11.777396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f22d7e74-ecf7-4829-9544-078721e25b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:11.783301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:11.787860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfbe36-7614-44e2-883d-38025c39a00c admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:11.838121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1071/2178] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:32:11.847130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca569e-c91e-49e2-8c49-2eb40395801a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:11.847578 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d7e74-ecf7-4829-9544-078721e25b56 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:11.848136 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca569e-c91e-49e2-8c49-2eb40395801a admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:11.848419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca569e-c91e-49e2-8c49-2eb40395801a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:11.848733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca569e-c91e-49e2-8c49-2eb40395801a admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:11.849116 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f22d7e74-ecf7-4829-9544-078721e25b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:11.849512 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca569e-c91e-49e2-8c49-2eb40395801a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:11.851456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca569e-c91e-49e2-8c49-2eb40395801a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:11.883055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1108/2179] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:32:11 2022] POST /identity/v3/projects => generated 361 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:32:11.884120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca569e-c91e-49e2-8c49-2eb40395801a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:11.899073 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:11.913424 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d7e74-ecf7-4829-9544-078721e25b56 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:11.961574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca569e-c91e-49e2-8c49-2eb40395801a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:11.966955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebde42-c03c-4226-aab7-aa197ad58f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:11.967631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebde42-c03c-4226-aab7-aa197ad58f5c admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:11.967906 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebde42-c03c-4226-aab7-aa197ad58f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:11.968208 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebde42-c03c-4226-aab7-aa197ad58f5c admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:11.968887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebde42-c03c-4226-aab7-aa197ad58f5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:11.970878 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebde42-c03c-4226-aab7-aa197ad58f5c admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:11.981167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24494f59-8eb5-44e2-9239-89ba8d370612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:11.982021 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24494f59-8eb5-44e2-9239-89ba8d370612 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:11.982345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24494f59-8eb5-44e2-9239-89ba8d370612 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:11.982623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24494f59-8eb5-44e2-9239-89ba8d370612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:11.993417 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24494f59-8eb5-44e2-9239-89ba8d370612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:11.994847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24494f59-8eb5-44e2-9239-89ba8d370612 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:12.006069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1109/2180] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:32:11 2022] POST /identity/v3/users => generated 281 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:32:12.006604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1072/2181] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:32:12.010157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebde42-c03c-4226-aab7-aa197ad58f5c admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:12.021217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebde42-c03c-4226-aab7-aa197ad58f5c admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:12.027431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74009d81-88dc-487c-8a45-bbdc91bf47d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:12.028489 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74009d81-88dc-487c-8a45-bbdc91bf47d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:12.028844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74009d81-88dc-487c-8a45-bbdc91bf47d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:12.029201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74009d81-88dc-487c-8a45-bbdc91bf47d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:12.030045 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74009d81-88dc-487c-8a45-bbdc91bf47d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:12.031897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74009d81-88dc-487c-8a45-bbdc91bf47d8 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:12.044680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1110/2182] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:32:12.055895 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24494f59-8eb5-44e2-9239-89ba8d370612 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:12.070779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24494f59-8eb5-44e2-9239-89ba8d370612 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:12.077250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f644857-4b6a-40a1-875f-e570309ae074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:12.078687 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f644857-4b6a-40a1-875f-e570309ae074 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:12.079117 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f644857-4b6a-40a1-875f-e570309ae074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:12.079688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f644857-4b6a-40a1-875f-e570309ae074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:12.080715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f644857-4b6a-40a1-875f-e570309ae074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:12.082561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f644857-4b6a-40a1-875f-e570309ae074 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:12.097187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1073/2183] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:32:12.101490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74009d81-88dc-487c-8a45-bbdc91bf47d8 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:12.112857 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74009d81-88dc-487c-8a45-bbdc91bf47d8 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:12.120170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3a5e2-e830-4556-a8db-bf3b2af49917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:12.121911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3a5e2-e830-4556-a8db-bf3b2af49917 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:12.122435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3a5e2-e830-4556-a8db-bf3b2af49917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:12.122775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3a5e2-e830-4556-a8db-bf3b2af49917 admin admin] PATH_INFO: `/v3/projects/f867b58cdfa74cc387dbdd4316a4f72b/users/9511b37f87414459bb2601f758fc48b5/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:12.140414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3a5e2-e830-4556-a8db-bf3b2af49917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f867b58cdfa74cc387dbdd4316a4f72b, user_id=9511b37f87414459bb2601f758fc48b5, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:12.143303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3a5e2-e830-4556-a8db-bf3b2af49917 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:12.159152 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1111/2184] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:32:12 2022] PUT /identity/v3/projects/f867b58cdfa74cc387dbdd4316a4f72b/users/9511b37f87414459bb2601f758fc48b5/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:32:12.177782 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4bb4e-b38a-40fa-a6dd-640ac22e6775 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:12.178029 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4bb4e-b38a-40fa-a6dd-640ac22e6775 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:12.178483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4bb4e-b38a-40fa-a6dd-640ac22e6775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:12.215625 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-57c4bb4e-b38a-40fa-a6dd-640ac22e6775 None None] MFA Rules not processed for user `9511b37f87414459bb2601f758fc48b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:12.217193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-57c4bb4e-b38a-40fa-a6dd-640ac22e6775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:12.303118 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1074/2185] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:12 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:12.327711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3a5e2-e830-4556-a8db-bf3b2af49917 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:12.330559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-78c3a5e2-e830-4556-a8db-bf3b2af49917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:12.338653 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4bb4e-b38a-40fa-a6dd-640ac22e6775 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:12.340345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-57c4bb4e-b38a-40fa-a6dd-640ac22e6775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:12.389765 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:12.409393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4bb4e-b38a-40fa-a6dd-640ac22e6775 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:12.409610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3a5e2-e830-4556-a8db-bf3b2af49917 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:12.414027 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebeff8-7421-4bd8-a0e9-0d4739218f20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:12.414686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebeff8-7421-4bd8-a0e9-0d4739218f20 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:12.414969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebeff8-7421-4bd8-a0e9-0d4739218f20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:12.415300 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebeff8-7421-4bd8-a0e9-0d4739218f20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:12.422026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63d846-b1df-46ad-a834-edc2f8d8b3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:12.422942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63d846-b1df-46ad-a834-edc2f8d8b3d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:12.423230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63d846-b1df-46ad-a834-edc2f8d8b3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:12.423509 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63d846-b1df-46ad-a834-edc2f8d8b3d0 admin admin] PATH_INFO: `/v3/projects/3253a3c2fb474da781ef0bff7b4a0b75` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:12.424732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63d846-b1df-46ad-a834-edc2f8d8b3d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3253a3c2fb474da781ef0bff7b4a0b75)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:12.425453 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebeff8-7421-4bd8-a0e9-0d4739218f20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:12.426717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63d846-b1df-46ad-a834-edc2f8d8b3d0 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:12.427125 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebeff8-7421-4bd8-a0e9-0d4739218f20 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:12.438151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1075/2186] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:32:12.453339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebeff8-7421-4bd8-a0e9-0d4739218f20 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:12.466354 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebeff8-7421-4bd8-a0e9-0d4739218f20 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:12.479038 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5f63d846-b1df-46ad-a834-edc2f8d8b3d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3253a3c2fb474da781ef0bff7b4a0b75', 'request_id': 'req-5f63d846-b1df-46ad-a834-edc2f8d8b3d0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:12.484812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5f63d846-b1df-46ad-a834-edc2f8d8b3d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3253a3c2fb474da781ef0bff7b4a0b75', 'request_id': 'req-5f63d846-b1df-46ad-a834-edc2f8d8b3d0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:12.488176 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:12.491836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5f63d846-b1df-46ad-a834-edc2f8d8b3d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3253a3c2fb474da781ef0bff7b4a0b75', 'request_id': 'req-5f63d846-b1df-46ad-a834-edc2f8d8b3d0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:12.507162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5f63d846-b1df-46ad-a834-edc2f8d8b3d0 admin admin] The token cache is being invalidate because project 3253a3c2fb474da781ef0bff7b4a0b75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:32:12.507458 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5f63d846-b1df-46ad-a834-edc2f8d8b3d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:12.508124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5f63d846-b1df-46ad-a834-edc2f8d8b3d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:12.510075 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1112/2187] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:32:12 2022] DELETE /identity/v3/projects/3253a3c2fb474da781ef0bff7b4a0b75 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:12.541464 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1fa24-32a2-4ab1-95ae-b1a140b287f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:12.542387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1fa24-32a2-4ab1-95ae-b1a140b287f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:12.542687 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1fa24-32a2-4ab1-95ae-b1a140b287f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:12.542951 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1fa24-32a2-4ab1-95ae-b1a140b287f6 admin admin] PATH_INFO: `/v3/users/9511b37f87414459bb2601f758fc48b5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:12.544352 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1fa24-32a2-4ab1-95ae-b1a140b287f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9511b37f87414459bb2601f758fc48b5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:12.546652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1fa24-32a2-4ab1-95ae-b1a140b287f6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:12.617472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4ce1fa24-32a2-4ab1-95ae-b1a140b287f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9511b37f87414459bb2601f758fc48b5'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:12.632401 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4ce1fa24-32a2-4ab1-95ae-b1a140b287f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9511b37f87414459bb2601f758fc48b5'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:12.633322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4ce1fa24-32a2-4ab1-95ae-b1a140b287f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9511b37f87414459bb2601f758fc48b5'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:12.634015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4ce1fa24-32a2-4ab1-95ae-b1a140b287f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9511b37f87414459bb2601f758fc48b5'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:12.642274 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4ce1fa24-32a2-4ab1-95ae-b1a140b287f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9511b37f87414459bb2601f758fc48b5'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:12.656710 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1076/2188] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:32:12 2022] DELETE /identity/v3/users/9511b37f87414459bb2601f758fc48b5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:12.661380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c215f-de83-4b6d-821b-802b99a84057 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:12.661677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c215f-de83-4b6d-821b-802b99a84057 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:12.661988 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c215f-de83-4b6d-821b-802b99a84057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:12.682732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-379c215f-de83-4b6d-821b-802b99a84057 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:12.683772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-379c215f-de83-4b6d-821b-802b99a84057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:12.746354 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1113/2189] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:12.756737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1fa24-32a2-4ab1-95ae-b1a140b287f6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:12.758156 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4ce1fa24-32a2-4ab1-95ae-b1a140b287f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:12.815351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1fa24-32a2-4ab1-95ae-b1a140b287f6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:12.819725 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b36f9-3d43-4da5-bc5c-27c33df1b5ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:12.820383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b36f9-3d43-4da5-bc5c-27c33df1b5ac service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:12.820649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b36f9-3d43-4da5-bc5c-27c33df1b5ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:12.820923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b36f9-3d43-4da5-bc5c-27c33df1b5ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:12.829856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b36f9-3d43-4da5-bc5c-27c33df1b5ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:12.831270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b36f9-3d43-4da5-bc5c-27c33df1b5ac service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:12.840893 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1077/2190] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:32:13.110862 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c215f-de83-4b6d-821b-802b99a84057 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:13.112336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-379c215f-de83-4b6d-821b-802b99a84057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:13.126099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c215f-de83-4b6d-821b-802b99a84057 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:13.131519 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8e7e8-b868-4309-87bb-3f13302cec99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:13.132428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8e7e8-b868-4309-87bb-3f13302cec99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:13.132708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8e7e8-b868-4309-87bb-3f13302cec99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:13.133028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8e7e8-b868-4309-87bb-3f13302cec99 admin admin] PATH_INFO: `/v3/projects/f867b58cdfa74cc387dbdd4316a4f72b` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:13.134334 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8e7e8-b868-4309-87bb-3f13302cec99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f867b58cdfa74cc387dbdd4316a4f72b)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:13.136267 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8e7e8-b868-4309-87bb-3f13302cec99 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:13.185049 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d5a8e7e8-b868-4309-87bb-3f13302cec99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f867b58cdfa74cc387dbdd4316a4f72b', 'request_id': 'req-d5a8e7e8-b868-4309-87bb-3f13302cec99'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:13.190265 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d5a8e7e8-b868-4309-87bb-3f13302cec99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f867b58cdfa74cc387dbdd4316a4f72b', 'request_id': 'req-d5a8e7e8-b868-4309-87bb-3f13302cec99'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:13.196266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d5a8e7e8-b868-4309-87bb-3f13302cec99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f867b58cdfa74cc387dbdd4316a4f72b', 'request_id': 'req-d5a8e7e8-b868-4309-87bb-3f13302cec99'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:13.204428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:13.210177 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d5a8e7e8-b868-4309-87bb-3f13302cec99 admin admin] The token cache is being invalidate because project f867b58cdfa74cc387dbdd4316a4f72b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:32:13.210504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d5a8e7e8-b868-4309-87bb-3f13302cec99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:13.211486 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-d5a8e7e8-b868-4309-87bb-3f13302cec99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:13.213132 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1114/2191] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:32:13 2022] DELETE /identity/v3/projects/f867b58cdfa74cc387dbdd4316a4f72b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:13.217924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c249bda-133d-4127-b51f-9a5d46239984 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:13.218210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c249bda-133d-4127-b51f-9a5d46239984 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:13.218483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c249bda-133d-4127-b51f-9a5d46239984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:13.236470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0c249bda-133d-4127-b51f-9a5d46239984 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:13.237713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0c249bda-133d-4127-b51f-9a5d46239984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:13.280508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:13.301351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1078/2192] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:13.309810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d30db7-23a3-40d3-89cc-8d5f5311dd5d None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:13.310098 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d30db7-23a3-40d3-89cc-8d5f5311dd5d None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:13.310377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d30db7-23a3-40d3-89cc-8d5f5311dd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:13.328786 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-76d30db7-23a3-40d3-89cc-8d5f5311dd5d None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:13.330012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-76d30db7-23a3-40d3-89cc-8d5f5311dd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:13.338337 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1115/2193] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:13.342404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c249bda-133d-4127-b51f-9a5d46239984 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:13.352744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c249bda-133d-4127-b51f-9a5d46239984 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:13.357334 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fae04f-0508-4d06-a692-dc7c3bca8195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:13.357983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fae04f-0508-4d06-a692-dc7c3bca8195 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:13.358267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fae04f-0508-4d06-a692-dc7c3bca8195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:13.358541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fae04f-0508-4d06-a692-dc7c3bca8195 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:13.359269 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fae04f-0508-4d06-a692-dc7c3bca8195 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:32:13.359575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fae04f-0508-4d06-a692-dc7c3bca8195 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:13.361186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fae04f-0508-4d06-a692-dc7c3bca8195 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:13.373318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1079/2194] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:32:13.377432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d30db7-23a3-40d3-89cc-8d5f5311dd5d None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:13.388418 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d30db7-23a3-40d3-89cc-8d5f5311dd5d None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:13.393400 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f376b-7952-43e7-8096-2d7f274b06d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:13.393760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f376b-7952-43e7-8096-2d7f274b06d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:13.394042 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f376b-7952-43e7-8096-2d7f274b06d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:13.394317 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f376b-7952-43e7-8096-2d7f274b06d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:13.395000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f376b-7952-43e7-8096-2d7f274b06d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:13.396937 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f376b-7952-43e7-8096-2d7f274b06d9 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:13.426009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1116/2195] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:32:13 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:32:13.430392 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fae04f-0508-4d06-a692-dc7c3bca8195 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:13.440464 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fae04f-0508-4d06-a692-dc7c3bca8195 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:13.498330 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94920f-a984-4410-bf99-2b8f839af7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:13.499004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94920f-a984-4410-bf99-2b8f839af7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:13.499291 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94920f-a984-4410-bf99-2b8f839af7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:13.499570 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94920f-a984-4410-bf99-2b8f839af7f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:13.500275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94920f-a984-4410-bf99-2b8f839af7f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:13.502251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94920f-a984-4410-bf99-2b8f839af7f8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:13.523329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1080/2196] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:32:13 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:32:13.527813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f376b-7952-43e7-8096-2d7f274b06d9 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:13.538909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f376b-7952-43e7-8096-2d7f274b06d9 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:13.543464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855a15e-e899-4ad5-b562-fcac80b47537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:13.544115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855a15e-e899-4ad5-b562-fcac80b47537 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:13.544405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855a15e-e899-4ad5-b562-fcac80b47537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:13.544682 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855a15e-e899-4ad5-b562-fcac80b47537 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:13.545446 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855a15e-e899-4ad5-b562-fcac80b47537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:13.547052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855a15e-e899-4ad5-b562-fcac80b47537 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:13.557283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1117/2197] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:32:13.561284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94920f-a984-4410-bf99-2b8f839af7f8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:13.571824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94920f-a984-4410-bf99-2b8f839af7f8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:13.576400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eecc7f-8c09-4bc6-8ff8-5cca496ceaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:13.577054 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eecc7f-8c09-4bc6-8ff8-5cca496ceaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:13.577364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eecc7f-8c09-4bc6-8ff8-5cca496ceaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:13.577658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eecc7f-8c09-4bc6-8ff8-5cca496ceaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:13.578380 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eecc7f-8c09-4bc6-8ff8-5cca496ceaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:13.579969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eecc7f-8c09-4bc6-8ff8-5cca496ceaa1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:13.590350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1081/2198] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:32:13.594296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855a15e-e899-4ad5-b562-fcac80b47537 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:13.604694 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855a15e-e899-4ad5-b562-fcac80b47537 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:13.609412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ac494-5907-456a-bd2a-817eb4b51147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:13.610296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ac494-5907-456a-bd2a-817eb4b51147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:13.610581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ac494-5907-456a-bd2a-817eb4b51147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:13.610859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ac494-5907-456a-bd2a-817eb4b51147 admin admin] PATH_INFO: `/v3/projects/9425e51c3abd4c8dbc9a6de5c2f438c7/users/364ee8b6b75d495ca3dc60916daa0afc/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:13.625620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ac494-5907-456a-bd2a-817eb4b51147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9425e51c3abd4c8dbc9a6de5c2f438c7, user_id=364ee8b6b75d495ca3dc60916daa0afc, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:13.628300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ac494-5907-456a-bd2a-817eb4b51147 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:13.642737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1118/2199] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:32:13 2022] PUT /identity/v3/projects/9425e51c3abd4c8dbc9a6de5c2f438c7/users/364ee8b6b75d495ca3dc60916daa0afc/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:32:13.655228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc128c-5f3e-42e9-99c1-3e9eb920428e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:13.655539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc128c-5f3e-42e9-99c1-3e9eb920428e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:13.655820 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc128c-5f3e-42e9-99c1-3e9eb920428e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:13.684482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-05bc128c-5f3e-42e9-99c1-3e9eb920428e None None] MFA Rules not processed for user `364ee8b6b75d495ca3dc60916daa0afc`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:13.685769 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-05bc128c-5f3e-42e9-99c1-3e9eb920428e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:13.765654 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1082/2200] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:13 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:13.773974 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ac494-5907-456a-bd2a-817eb4b51147 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:13.775707 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-552ac494-5907-456a-bd2a-817eb4b51147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:13.834157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ac494-5907-456a-bd2a-817eb4b51147 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:13.838283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d2e77-e53b-422e-a4c7-a0cbef597174 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:13.838943 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d2e77-e53b-422e-a4c7-a0cbef597174 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:13.839230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d2e77-e53b-422e-a4c7-a0cbef597174 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:13.839505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d2e77-e53b-422e-a4c7-a0cbef597174 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:13.848176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d2e77-e53b-422e-a4c7-a0cbef597174 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:13.850013 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d2e77-e53b-422e-a4c7-a0cbef597174 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:13.859022 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1119/2201] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:13 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:32:15.566577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc128c-5f3e-42e9-99c1-3e9eb920428e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:15.568539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-05bc128c-5f3e-42e9-99c1-3e9eb920428e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:15.627364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:15.656863 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc128c-5f3e-42e9-99c1-3e9eb920428e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:15.662173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624de2a5-1c34-4a00-9308-667fdb924557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:15.663266 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624de2a5-1c34-4a00-9308-667fdb924557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:15.663643 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624de2a5-1c34-4a00-9308-667fdb924557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:15.664047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624de2a5-1c34-4a00-9308-667fdb924557 admin admin] PATH_INFO: `/v3/users/fd3dc5dde61d411d82a962b40feecb5b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:15.666450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624de2a5-1c34-4a00-9308-667fdb924557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd3dc5dde61d411d82a962b40feecb5b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:15.668481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624de2a5-1c34-4a00-9308-667fdb924557 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:15.740692 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-624de2a5-1c34-4a00-9308-667fdb924557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd3dc5dde61d411d82a962b40feecb5b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:15.759076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-624de2a5-1c34-4a00-9308-667fdb924557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd3dc5dde61d411d82a962b40feecb5b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:15.759889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-624de2a5-1c34-4a00-9308-667fdb924557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd3dc5dde61d411d82a962b40feecb5b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:15.760688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-624de2a5-1c34-4a00-9308-667fdb924557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd3dc5dde61d411d82a962b40feecb5b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:15.769337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-624de2a5-1c34-4a00-9308-667fdb924557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd3dc5dde61d411d82a962b40feecb5b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:15.783709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1083/2202] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:32:15 2022] DELETE /identity/v3/users/fd3dc5dde61d411d82a962b40feecb5b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:15.788527 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42928d4d-b38d-40a3-b550-aac9c247ff7d None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:15.788940 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42928d4d-b38d-40a3-b550-aac9c247ff7d None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:15.789124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42928d4d-b38d-40a3-b550-aac9c247ff7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:15.808155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-42928d4d-b38d-40a3-b550-aac9c247ff7d None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:15.809387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-42928d4d-b38d-40a3-b550-aac9c247ff7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:15.876207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1120/2203] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:15.885894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624de2a5-1c34-4a00-9308-667fdb924557 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:15.887283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-624de2a5-1c34-4a00-9308-667fdb924557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:15.957222 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624de2a5-1c34-4a00-9308-667fdb924557 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:15.960939 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373bb25-4123-4009-9e33-d1ac93678619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:15.961629 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373bb25-4123-4009-9e33-d1ac93678619 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:15.961915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373bb25-4123-4009-9e33-d1ac93678619 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:15.962195 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373bb25-4123-4009-9e33-d1ac93678619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:15.971388 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373bb25-4123-4009-9e33-d1ac93678619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:15.972820 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373bb25-4123-4009-9e33-d1ac93678619 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:15.982713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1084/2204] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:32:16.333905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42928d4d-b38d-40a3-b550-aac9c247ff7d None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:16.337095 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-42928d4d-b38d-40a3-b550-aac9c247ff7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:16.350309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42928d4d-b38d-40a3-b550-aac9c247ff7d None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:16.355163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f2523-062b-4630-94dc-ace3c502a807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:16.356085 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f2523-062b-4630-94dc-ace3c502a807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:16.356367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f2523-062b-4630-94dc-ace3c502a807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:16.356653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f2523-062b-4630-94dc-ace3c502a807 admin admin] PATH_INFO: `/v3/projects/8d4c2a62dfbf4c7e99fea8837ad85749` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:16.357902 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f2523-062b-4630-94dc-ace3c502a807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d4c2a62dfbf4c7e99fea8837ad85749)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:16.359821 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f2523-062b-4630-94dc-ace3c502a807 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:16.395511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:16.409442 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-496f2523-062b-4630-94dc-ace3c502a807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d4c2a62dfbf4c7e99fea8837ad85749', 'request_id': 'req-496f2523-062b-4630-94dc-ace3c502a807'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:16.415990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-496f2523-062b-4630-94dc-ace3c502a807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d4c2a62dfbf4c7e99fea8837ad85749', 'request_id': 'req-496f2523-062b-4630-94dc-ace3c502a807'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:16.422284 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-496f2523-062b-4630-94dc-ace3c502a807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d4c2a62dfbf4c7e99fea8837ad85749', 'request_id': 'req-496f2523-062b-4630-94dc-ace3c502a807'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:16.435690 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-496f2523-062b-4630-94dc-ace3c502a807 admin admin] The token cache is being invalidate because project 8d4c2a62dfbf4c7e99fea8837ad85749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:32:16.436004 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-496f2523-062b-4630-94dc-ace3c502a807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:16.436617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-496f2523-062b-4630-94dc-ace3c502a807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:16.438238 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1121/2205] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:32:16 2022] DELETE /identity/v3/projects/8d4c2a62dfbf4c7e99fea8837ad85749 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:16.442238 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373bb25-4123-4009-9e33-d1ac93678619 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:16.452748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373bb25-4123-4009-9e33-d1ac93678619 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:16.509063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800f08a-ddfd-4663-ac91-d6ae17c76f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:16.509973 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800f08a-ddfd-4663-ac91-d6ae17c76f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:16.510255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800f08a-ddfd-4663-ac91-d6ae17c76f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:16.510529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800f08a-ddfd-4663-ac91-d6ae17c76f9c admin admin] PATH_INFO: `/v3/users/3a7cfaa194804f1e94ac12971f935fc4` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:16.511709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800f08a-ddfd-4663-ac91-d6ae17c76f9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a7cfaa194804f1e94ac12971f935fc4)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:16.513671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800f08a-ddfd-4663-ac91-d6ae17c76f9c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:16.515902 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:16.561642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6800f08a-ddfd-4663-ac91-d6ae17c76f9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a7cfaa194804f1e94ac12971f935fc4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:16.573695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6800f08a-ddfd-4663-ac91-d6ae17c76f9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a7cfaa194804f1e94ac12971f935fc4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:16.574415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6800f08a-ddfd-4663-ac91-d6ae17c76f9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a7cfaa194804f1e94ac12971f935fc4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:16.575020 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6800f08a-ddfd-4663-ac91-d6ae17c76f9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a7cfaa194804f1e94ac12971f935fc4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:16.581956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6800f08a-ddfd-4663-ac91-d6ae17c76f9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a7cfaa194804f1e94ac12971f935fc4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:16.594978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1085/2206] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:32:16 2022] DELETE /identity/v3/users/3a7cfaa194804f1e94ac12971f935fc4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:32:16.916086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f2523-062b-4630-94dc-ace3c502a807 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:16.917612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-496f2523-062b-4630-94dc-ace3c502a807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:16.936288 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f2523-062b-4630-94dc-ace3c502a807 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:17.022094 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c46a2-d623-41cb-96c6-c14275f1b786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:17.023172 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c46a2-d623-41cb-96c6-c14275f1b786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:17.023441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c46a2-d623-41cb-96c6-c14275f1b786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:17.024013 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c46a2-d623-41cb-96c6-c14275f1b786 admin admin] PATH_INFO: `/v3/projects/a03de89f20d54a57b8c02c45ded8d3b8` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:17.025233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c46a2-d623-41cb-96c6-c14275f1b786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a03de89f20d54a57b8c02c45ded8d3b8)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:17.028211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c46a2-d623-41cb-96c6-c14275f1b786 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:17.126646 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ed5c46a2-d623-41cb-96c6-c14275f1b786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a03de89f20d54a57b8c02c45ded8d3b8', 'request_id': 'req-ed5c46a2-d623-41cb-96c6-c14275f1b786'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:17.143085 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ed5c46a2-d623-41cb-96c6-c14275f1b786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a03de89f20d54a57b8c02c45ded8d3b8', 'request_id': 'req-ed5c46a2-d623-41cb-96c6-c14275f1b786'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:17.155738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ed5c46a2-d623-41cb-96c6-c14275f1b786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a03de89f20d54a57b8c02c45ded8d3b8', 'request_id': 'req-ed5c46a2-d623-41cb-96c6-c14275f1b786'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:17.179412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ed5c46a2-d623-41cb-96c6-c14275f1b786 admin admin] The token cache is being invalidate because project a03de89f20d54a57b8c02c45ded8d3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:32:17.179755 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ed5c46a2-d623-41cb-96c6-c14275f1b786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:17.182571 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ed5c46a2-d623-41cb-96c6-c14275f1b786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:17.187409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1122/2207] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:32:16 2022] DELETE /identity/v3/projects/a03de89f20d54a57b8c02c45ded8d3b8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:17.190661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800f08a-ddfd-4663-ac91-d6ae17c76f9c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:17.192031 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6800f08a-ddfd-4663-ac91-d6ae17c76f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:17.219083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800f08a-ddfd-4663-ac91-d6ae17c76f9c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:17.325637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9284d1-833b-409e-8fb0-9b52804c2cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:17.326532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9284d1-833b-409e-8fb0-9b52804c2cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:17.326815 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9284d1-833b-409e-8fb0-9b52804c2cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:17.327086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9284d1-833b-409e-8fb0-9b52804c2cb3 admin admin] PATH_INFO: `/v3/users/ef2228b364b14c8693df3bbd2fada5b4` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:17.328290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9284d1-833b-409e-8fb0-9b52804c2cb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef2228b364b14c8693df3bbd2fada5b4)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:17.330270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9284d1-833b-409e-8fb0-9b52804c2cb3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:17.380486 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-de9284d1-833b-409e-8fb0-9b52804c2cb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef2228b364b14c8693df3bbd2fada5b4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:17.392187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-de9284d1-833b-409e-8fb0-9b52804c2cb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef2228b364b14c8693df3bbd2fada5b4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:17.392861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-de9284d1-833b-409e-8fb0-9b52804c2cb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef2228b364b14c8693df3bbd2fada5b4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:17.393483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-de9284d1-833b-409e-8fb0-9b52804c2cb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef2228b364b14c8693df3bbd2fada5b4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:17.401166 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-de9284d1-833b-409e-8fb0-9b52804c2cb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef2228b364b14c8693df3bbd2fada5b4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:17.409974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:17.416083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1086/2208] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:32:17 2022] DELETE /identity/v3/users/ef2228b364b14c8693df3bbd2fada5b4 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:17.688325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c46a2-d623-41cb-96c6-c14275f1b786 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:17.689897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ed5c46a2-d623-41cb-96c6-c14275f1b786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:17.705271 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c46a2-d623-41cb-96c6-c14275f1b786 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:17.754063 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:17.766444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26d4ae-e799-49df-868d-db940c1ecc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:17.767245 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26d4ae-e799-49df-868d-db940c1ecc1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:17.767526 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26d4ae-e799-49df-868d-db940c1ecc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:17.767800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26d4ae-e799-49df-868d-db940c1ecc1d admin admin] PATH_INFO: `/v3/projects/caf9021f4fde43c68357c9a595cc4c5f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:17.768985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26d4ae-e799-49df-868d-db940c1ecc1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=caf9021f4fde43c68357c9a595cc4c5f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:17.771691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26d4ae-e799-49df-868d-db940c1ecc1d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:17.815951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ce26d4ae-e799-49df-868d-db940c1ecc1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caf9021f4fde43c68357c9a595cc4c5f', 'request_id': 'req-ce26d4ae-e799-49df-868d-db940c1ecc1d'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:17.822012 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ce26d4ae-e799-49df-868d-db940c1ecc1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caf9021f4fde43c68357c9a595cc4c5f', 'request_id': 'req-ce26d4ae-e799-49df-868d-db940c1ecc1d'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:17.828000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ce26d4ae-e799-49df-868d-db940c1ecc1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caf9021f4fde43c68357c9a595cc4c5f', 'request_id': 'req-ce26d4ae-e799-49df-868d-db940c1ecc1d'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:17.840856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ce26d4ae-e799-49df-868d-db940c1ecc1d admin admin] The token cache is being invalidate because project caf9021f4fde43c68357c9a595cc4c5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:32:17.841158 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ce26d4ae-e799-49df-868d-db940c1ecc1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:17.841790 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ce26d4ae-e799-49df-868d-db940c1ecc1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:17.843343 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1123/2209] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:32:17 2022] DELETE /identity/v3/projects/caf9021f4fde43c68357c9a595cc4c5f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:27.838015 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:28.325074 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9284d1-833b-409e-8fb0-9b52804c2cb3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:28.326803 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-de9284d1-833b-409e-8fb0-9b52804c2cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:28.341179 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9284d1-833b-409e-8fb0-9b52804c2cb3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:28.400543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9766f6-c46b-4d59-a2e6-4cf6f391cb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:28.401633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9766f6-c46b-4d59-a2e6-4cf6f391cb03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:28.402016 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9766f6-c46b-4d59-a2e6-4cf6f391cb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:28.402377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9766f6-c46b-4d59-a2e6-4cf6f391cb03 admin admin] PATH_INFO: `/v3/users/f1e4c061613b4c1fa53b4ffbbb4a8d89` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:28.403870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9766f6-c46b-4d59-a2e6-4cf6f391cb03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e4c061613b4c1fa53b4ffbbb4a8d89)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:28.405838 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9766f6-c46b-4d59-a2e6-4cf6f391cb03 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:28.468346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fb9766f6-c46b-4d59-a2e6-4cf6f391cb03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e4c061613b4c1fa53b4ffbbb4a8d89'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:28.482140 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fb9766f6-c46b-4d59-a2e6-4cf6f391cb03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e4c061613b4c1fa53b4ffbbb4a8d89'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:28.482846 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fb9766f6-c46b-4d59-a2e6-4cf6f391cb03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e4c061613b4c1fa53b4ffbbb4a8d89'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:28.483441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fb9766f6-c46b-4d59-a2e6-4cf6f391cb03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e4c061613b4c1fa53b4ffbbb4a8d89'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:28.496299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fb9766f6-c46b-4d59-a2e6-4cf6f391cb03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e4c061613b4c1fa53b4ffbbb4a8d89'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:28.513862 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1087/2210] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:32:28 2022] DELETE /identity/v3/users/f1e4c061613b4c1fa53b4ffbbb4a8d89 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:28.527302 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428ad09-ed56-4678-889c-a83cc5f643e9 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:28.527623 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428ad09-ed56-4678-889c-a83cc5f643e9 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:28.527920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428ad09-ed56-4678-889c-a83cc5f643e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:28.545550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-5428ad09-ed56-4678-889c-a83cc5f643e9 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:28.546762 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5428ad09-ed56-4678-889c-a83cc5f643e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:28.606865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1124/2211] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:28.616967 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9766f6-c46b-4d59-a2e6-4cf6f391cb03 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:28.618394 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fb9766f6-c46b-4d59-a2e6-4cf6f391cb03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:28.675971 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9766f6-c46b-4d59-a2e6-4cf6f391cb03 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:28.680145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb5f34-cbee-427c-bc61-58060f226e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:28.680829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb5f34-cbee-427c-bc61-58060f226e0d service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:28.681167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb5f34-cbee-427c-bc61-58060f226e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:28.681450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb5f34-cbee-427c-bc61-58060f226e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:28.690598 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb5f34-cbee-427c-bc61-58060f226e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:28.692709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb5f34-cbee-427c-bc61-58060f226e0d service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:28.702263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1088/2212] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:32:28.972104 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428ad09-ed56-4678-889c-a83cc5f643e9 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:28.974143 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5428ad09-ed56-4678-889c-a83cc5f643e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:28.986236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428ad09-ed56-4678-889c-a83cc5f643e9 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:28.992485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88304c-180c-4110-a377-9934bef0930d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:28.995577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88304c-180c-4110-a377-9934bef0930d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:28.995577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88304c-180c-4110-a377-9934bef0930d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:28.995577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88304c-180c-4110-a377-9934bef0930d admin admin] PATH_INFO: `/v3/projects/210471bf1a9a494884ba685b7ccb5f67` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:28.995577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88304c-180c-4110-a377-9934bef0930d admin admin] RBAC: Authorizing `identity:delete_project(project_id=210471bf1a9a494884ba685b7ccb5f67)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:28.997079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88304c-180c-4110-a377-9934bef0930d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:29.053909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-3f88304c-180c-4110-a377-9934bef0930d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '210471bf1a9a494884ba685b7ccb5f67', 'request_id': 'req-3f88304c-180c-4110-a377-9934bef0930d'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:29.060375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-3f88304c-180c-4110-a377-9934bef0930d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '210471bf1a9a494884ba685b7ccb5f67', 'request_id': 'req-3f88304c-180c-4110-a377-9934bef0930d'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:29.066643 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-3f88304c-180c-4110-a377-9934bef0930d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '210471bf1a9a494884ba685b7ccb5f67', 'request_id': 'req-3f88304c-180c-4110-a377-9934bef0930d'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:29.080684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-3f88304c-180c-4110-a377-9934bef0930d admin admin] The token cache is being invalidate because project 210471bf1a9a494884ba685b7ccb5f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:32:29.080972 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-3f88304c-180c-4110-a377-9934bef0930d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:29.081636 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-3f88304c-180c-4110-a377-9934bef0930d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:29.083565 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1125/2213] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:32:28 2022] DELETE /identity/v3/projects/210471bf1a9a494884ba685b7ccb5f67 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:29.087456 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb5f34-cbee-427c-bc61-58060f226e0d service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:29.097364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb5f34-cbee-427c-bc61-58060f226e0d service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:29.116887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:29.157156 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1e1ba-1abd-46cd-a8f1-2cefcfaf662d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:29.158048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1e1ba-1abd-46cd-a8f1-2cefcfaf662d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:29.158331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1e1ba-1abd-46cd-a8f1-2cefcfaf662d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:29.158607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1e1ba-1abd-46cd-a8f1-2cefcfaf662d admin admin] PATH_INFO: `/v3/users/a07fa4a6e5ba41929c18ab5433451c8d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:29.159780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1e1ba-1abd-46cd-a8f1-2cefcfaf662d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a07fa4a6e5ba41929c18ab5433451c8d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:29.161827 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1e1ba-1abd-46cd-a8f1-2cefcfaf662d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:29.206623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-25f1e1ba-1abd-46cd-a8f1-2cefcfaf662d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a07fa4a6e5ba41929c18ab5433451c8d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:29.218230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-25f1e1ba-1abd-46cd-a8f1-2cefcfaf662d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a07fa4a6e5ba41929c18ab5433451c8d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:29.218880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-25f1e1ba-1abd-46cd-a8f1-2cefcfaf662d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a07fa4a6e5ba41929c18ab5433451c8d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:29.219481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-25f1e1ba-1abd-46cd-a8f1-2cefcfaf662d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a07fa4a6e5ba41929c18ab5433451c8d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:29.226765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-25f1e1ba-1abd-46cd-a8f1-2cefcfaf662d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a07fa4a6e5ba41929c18ab5433451c8d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:29.240123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1089/2214] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:32:29 2022] DELETE /identity/v3/users/a07fa4a6e5ba41929c18ab5433451c8d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:29.556730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88304c-180c-4110-a377-9934bef0930d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:29.558296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3f88304c-180c-4110-a377-9934bef0930d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:29.573471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88304c-180c-4110-a377-9934bef0930d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:29.657537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fa4ae-4518-4942-b305-51d68c87d5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:29.658416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fa4ae-4518-4942-b305-51d68c87d5cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:29.658701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fa4ae-4518-4942-b305-51d68c87d5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:29.658978 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fa4ae-4518-4942-b305-51d68c87d5cd admin admin] PATH_INFO: `/v3/projects/5d57435b3df34818a0e3c62339ff91b2` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:29.660208 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fa4ae-4518-4942-b305-51d68c87d5cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d57435b3df34818a0e3c62339ff91b2)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:29.662514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fa4ae-4518-4942-b305-51d68c87d5cd admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:29.697925 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:29.716017 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-216fa4ae-4518-4942-b305-51d68c87d5cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d57435b3df34818a0e3c62339ff91b2', 'request_id': 'req-216fa4ae-4518-4942-b305-51d68c87d5cd'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:29.722264 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-216fa4ae-4518-4942-b305-51d68c87d5cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d57435b3df34818a0e3c62339ff91b2', 'request_id': 'req-216fa4ae-4518-4942-b305-51d68c87d5cd'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:29.729069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-216fa4ae-4518-4942-b305-51d68c87d5cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d57435b3df34818a0e3c62339ff91b2', 'request_id': 'req-216fa4ae-4518-4942-b305-51d68c87d5cd'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:29.742806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-216fa4ae-4518-4942-b305-51d68c87d5cd admin admin] The token cache is being invalidate because project 5d57435b3df34818a0e3c62339ff91b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:32:29.743126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-216fa4ae-4518-4942-b305-51d68c87d5cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:29.743762 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-216fa4ae-4518-4942-b305-51d68c87d5cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:29.746580 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1126/2215] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:32:29 2022] DELETE /identity/v3/projects/5d57435b3df34818a0e3c62339ff91b2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:29.750226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5dec5-19c5-4a09-9e62-881db322a347 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:29.750700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5dec5-19c5-4a09-9e62-881db322a347 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:29.750792 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5dec5-19c5-4a09-9e62-881db322a347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:29.770722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1ff5dec5-19c5-4a09-9e62-881db322a347 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:29.772203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1ff5dec5-19c5-4a09-9e62-881db322a347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:29.833881 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1090/2216] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:32:29.842343 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84704d1-c49c-4800-b229-d2e1f09f8035 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:29.842634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84704d1-c49c-4800-b229-d2e1f09f8035 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:29.842912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84704d1-c49c-4800-b229-d2e1f09f8035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:29.861719 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-d84704d1-c49c-4800-b229-d2e1f09f8035 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:29.862927 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d84704d1-c49c-4800-b229-d2e1f09f8035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:29.871391 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1127/2217] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:29.875678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5dec5-19c5-4a09-9e62-881db322a347 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:29.886581 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5dec5-19c5-4a09-9e62-881db322a347 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:29.891065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944fb16-e368-4310-bf3f-b22b37714692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:29.891722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944fb16-e368-4310-bf3f-b22b37714692 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:29.892002 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944fb16-e368-4310-bf3f-b22b37714692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:29.892277 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944fb16-e368-4310-bf3f-b22b37714692 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:29.893005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e944fb16-e368-4310-bf3f-b22b37714692 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:32:29.893537 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e944fb16-e368-4310-bf3f-b22b37714692 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:29.895976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e944fb16-e368-4310-bf3f-b22b37714692 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:29.909694 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1091/2218] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:32:29.914102 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84704d1-c49c-4800-b229-d2e1f09f8035 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:29.923601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84704d1-c49c-4800-b229-d2e1f09f8035 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:29.928905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801d2ae-260c-462c-b6e2-a9efab6bc7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:29.929630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801d2ae-260c-462c-b6e2-a9efab6bc7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:29.929933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801d2ae-260c-462c-b6e2-a9efab6bc7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:29.930220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801d2ae-260c-462c-b6e2-a9efab6bc7a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:29.931053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801d2ae-260c-462c-b6e2-a9efab6bc7a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:29.933024 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801d2ae-260c-462c-b6e2-a9efab6bc7a9 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:29.959962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1128/2219] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:32:29 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:32:29.964022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944fb16-e368-4310-bf3f-b22b37714692 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:29.975066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944fb16-e368-4310-bf3f-b22b37714692 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:30.002558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:30.036165 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce49e1-251f-4a7a-80c8-c8dc90bc0b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:30.036839 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce49e1-251f-4a7a-80c8-c8dc90bc0b90 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:30.037124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce49e1-251f-4a7a-80c8-c8dc90bc0b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:30.037425 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce49e1-251f-4a7a-80c8-c8dc90bc0b90 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:30.038124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce49e1-251f-4a7a-80c8-c8dc90bc0b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:30.040044 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce49e1-251f-4a7a-80c8-c8dc90bc0b90 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:30.060051 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1092/2220] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:32:29 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:32:30.064149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801d2ae-260c-462c-b6e2-a9efab6bc7a9 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:30.074749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801d2ae-260c-462c-b6e2-a9efab6bc7a9 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:30.079374 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d9d26-f691-41e8-820a-eb64ed432ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:30.080031 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d9d26-f691-41e8-820a-eb64ed432ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:30.080801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d9d26-f691-41e8-820a-eb64ed432ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:30.080801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d9d26-f691-41e8-820a-eb64ed432ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:30.081366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d9d26-f691-41e8-820a-eb64ed432ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:30.083935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d9d26-f691-41e8-820a-eb64ed432ff1 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:30.094328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1129/2221] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:32:30.099047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce49e1-251f-4a7a-80c8-c8dc90bc0b90 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:30.109370 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce49e1-251f-4a7a-80c8-c8dc90bc0b90 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:30.114391 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d8c39-5d15-423f-adae-726b6001fad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:30.115074 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d8c39-5d15-423f-adae-726b6001fad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:30.115384 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d8c39-5d15-423f-adae-726b6001fad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:30.115663 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d8c39-5d15-423f-adae-726b6001fad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:30.116400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d8c39-5d15-423f-adae-726b6001fad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:30.118638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d8c39-5d15-423f-adae-726b6001fad5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:30.129780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1093/2222] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:32:30.133944 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d9d26-f691-41e8-820a-eb64ed432ff1 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:30.144133 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d9d26-f691-41e8-820a-eb64ed432ff1 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:30.150336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be02c5-3e8d-4eba-8a56-0e250771f069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:30.151306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be02c5-3e8d-4eba-8a56-0e250771f069 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:30.151658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be02c5-3e8d-4eba-8a56-0e250771f069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:30.152045 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be02c5-3e8d-4eba-8a56-0e250771f069 admin admin] PATH_INFO: `/v3/projects/fe9f05b7338d4b5d9c7478f9b542076f/users/b6680c63875d4f82a8a9dbe5ecaeb83b/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:30.168190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be02c5-3e8d-4eba-8a56-0e250771f069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9f05b7338d4b5d9c7478f9b542076f, user_id=b6680c63875d4f82a8a9dbe5ecaeb83b, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:30.171023 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be02c5-3e8d-4eba-8a56-0e250771f069 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:30.185453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1130/2223] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:32:30 2022] PUT /identity/v3/projects/fe9f05b7338d4b5d9c7478f9b542076f/users/b6680c63875d4f82a8a9dbe5ecaeb83b/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:30.196351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25202b-8d93-4fb4-9742-0207a234fed3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:30.196646 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25202b-8d93-4fb4-9742-0207a234fed3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:30.196936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25202b-8d93-4fb4-9742-0207a234fed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:30.224881 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9f25202b-8d93-4fb4-9742-0207a234fed3 None None] MFA Rules not processed for user `b6680c63875d4f82a8a9dbe5ecaeb83b`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:30.226125 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9f25202b-8d93-4fb4-9742-0207a234fed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:30.298016 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1094/2224] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:30.303082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be02c5-3e8d-4eba-8a56-0e250771f069 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:30.365894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be02c5-3e8d-4eba-8a56-0e250771f069 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:30.370528 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b5726-1e9a-43d0-a30d-61a631b9bb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:30.371187 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b5726-1e9a-43d0-a30d-61a631b9bb06 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:30.371480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b5726-1e9a-43d0-a30d-61a631b9bb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:30.371759 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b5726-1e9a-43d0-a30d-61a631b9bb06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:30.372445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b5726-1e9a-43d0-a30d-61a631b9bb06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:30.374414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b5726-1e9a-43d0-a30d-61a631b9bb06 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:30.399799 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1131/2225] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:32:30 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:32:30.403794 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25202b-8d93-4fb4-9742-0207a234fed3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:30.415151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25202b-8d93-4fb4-9742-0207a234fed3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:30.471763 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d22271-8df7-4b0a-81b6-2b4c0232418e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:30.472445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d22271-8df7-4b0a-81b6-2b4c0232418e admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:30.472726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d22271-8df7-4b0a-81b6-2b4c0232418e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:30.473608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d22271-8df7-4b0a-81b6-2b4c0232418e admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:30.474308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d22271-8df7-4b0a-81b6-2b4c0232418e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:30.476233 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d22271-8df7-4b0a-81b6-2b4c0232418e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:30.496150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1095/2226] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:32:30 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:32:30.500185 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b5726-1e9a-43d0-a30d-61a631b9bb06 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:30.510468 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b5726-1e9a-43d0-a30d-61a631b9bb06 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:30.515055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6add7a-33ad-42c3-b9e7-ec2f72a9ed44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:30.515686 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6add7a-33ad-42c3-b9e7-ec2f72a9ed44 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:30.515966 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6add7a-33ad-42c3-b9e7-ec2f72a9ed44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:30.516222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6add7a-33ad-42c3-b9e7-ec2f72a9ed44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:30.516952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6add7a-33ad-42c3-b9e7-ec2f72a9ed44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:30.518571 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6add7a-33ad-42c3-b9e7-ec2f72a9ed44 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:30.530457 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1132/2227] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:32:30.534707 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d22271-8df7-4b0a-81b6-2b4c0232418e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:30.545747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d22271-8df7-4b0a-81b6-2b4c0232418e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:30.550354 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2a0fa-1605-4734-9662-a6f36756a986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:30.551940 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2a0fa-1605-4734-9662-a6f36756a986 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:30.552256 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2a0fa-1605-4734-9662-a6f36756a986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:30.552673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2a0fa-1605-4734-9662-a6f36756a986 admin admin] PATH_INFO: `/v3/projects/d765911ee6174e1fb8b795a7e2dd46bd/users/023e2d492dca46f8b11c9e656a6170ce/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:30.569037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2a0fa-1605-4734-9662-a6f36756a986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d765911ee6174e1fb8b795a7e2dd46bd, user_id=023e2d492dca46f8b11c9e656a6170ce, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:30.571732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2a0fa-1605-4734-9662-a6f36756a986 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:30.586035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1096/2228] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:32:30 2022] PUT /identity/v3/projects/d765911ee6174e1fb8b795a7e2dd46bd/users/023e2d492dca46f8b11c9e656a6170ce/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:30.589852 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6add7a-33ad-42c3-b9e7-ec2f72a9ed44 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:30.600448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6add7a-33ad-42c3-b9e7-ec2f72a9ed44 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:30.620746 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:30.662174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fe3c4-569b-4e6b-96bf-3f93cc3f53ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:30.662870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fe3c4-569b-4e6b-96bf-3f93cc3f53ec admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:30.663282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fe3c4-569b-4e6b-96bf-3f93cc3f53ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:30.663637 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fe3c4-569b-4e6b-96bf-3f93cc3f53ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:30.664531 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fe3c4-569b-4e6b-96bf-3f93cc3f53ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:30.666253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fe3c4-569b-4e6b-96bf-3f93cc3f53ec admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:30.685830 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1133/2229] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:32:30.685982 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2a0fa-1605-4734-9662-a6f36756a986 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:30.688607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:30.695888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2a0fa-1605-4734-9662-a6f36756a986 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:30.701191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0d6b9-7164-47b6-9995-5ec07102e428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:30.702129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0d6b9-7164-47b6-9995-5ec07102e428 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:30.702412 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0d6b9-7164-47b6-9995-5ec07102e428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:30.702691 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0d6b9-7164-47b6-9995-5ec07102e428 admin admin] PATH_INFO: `/v3/projects/d765911ee6174e1fb8b795a7e2dd46bd/users/023e2d492dca46f8b11c9e656a6170ce/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:30.704700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0d6b9-7164-47b6-9995-5ec07102e428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d765911ee6174e1fb8b795a7e2dd46bd, user_id=023e2d492dca46f8b11c9e656a6170ce, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:30.707485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0d6b9-7164-47b6-9995-5ec07102e428 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:30.719020 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1097/2230] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:32:30 2022] PUT /identity/v3/projects/d765911ee6174e1fb8b795a7e2dd46bd/users/023e2d492dca46f8b11c9e656a6170ce/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:32:30.729954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b4d9d-a1d3-4ee8-b97f-65a448bcd954 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:30.730248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b4d9d-a1d3-4ee8-b97f-65a448bcd954 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:30.730530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b4d9d-a1d3-4ee8-b97f-65a448bcd954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:30.759718 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-274b4d9d-a1d3-4ee8-b97f-65a448bcd954 None None] MFA Rules not processed for user `023e2d492dca46f8b11c9e656a6170ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:30.761017 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-274b4d9d-a1d3-4ee8-b97f-65a448bcd954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:30.831662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1134/2231] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:30.843642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0d6b9-7164-47b6-9995-5ec07102e428 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:30.845417 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bcc0d6b9-7164-47b6-9995-5ec07102e428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:30.913609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0d6b9-7164-47b6-9995-5ec07102e428 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:30.917230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71caf4df-8ee6-4228-b740-bc8f9b112da5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:30.917884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71caf4df-8ee6-4228-b740-bc8f9b112da5 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:30.918171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71caf4df-8ee6-4228-b740-bc8f9b112da5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:30.918459 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71caf4df-8ee6-4228-b740-bc8f9b112da5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:30.928124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71caf4df-8ee6-4228-b740-bc8f9b112da5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:30.929607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71caf4df-8ee6-4228-b740-bc8f9b112da5 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:30.939231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1098/2232] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:30 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:32:30.980815 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b4d9d-a1d3-4ee8-b97f-65a448bcd954 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:30.982318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-274b4d9d-a1d3-4ee8-b97f-65a448bcd954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:30.995046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b4d9d-a1d3-4ee8-b97f-65a448bcd954 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:30.998746 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a78af8-96ee-4be2-b55c-bffd23338c17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:30.999417 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a78af8-96ee-4be2-b55c-bffd23338c17 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:30.999704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a78af8-96ee-4be2-b55c-bffd23338c17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:30.999985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a78af8-96ee-4be2-b55c-bffd23338c17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:31.060831 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a78af8-96ee-4be2-b55c-bffd23338c17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:31.062478 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a78af8-96ee-4be2-b55c-bffd23338c17 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:31.072748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1135/2233] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:32:34.385932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71caf4df-8ee6-4228-b740-bc8f9b112da5 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:34.451678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71caf4df-8ee6-4228-b740-bc8f9b112da5 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:34.456159 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b86aa-df16-4058-a399-d250cbc7c4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:34.457067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b86aa-df16-4058-a399-d250cbc7c4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:34.458131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b86aa-df16-4058-a399-d250cbc7c4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:34.458531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b86aa-df16-4058-a399-d250cbc7c4f9 admin admin] PATH_INFO: `/v3/users/b6680c63875d4f82a8a9dbe5ecaeb83b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:34.459720 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b86aa-df16-4058-a399-d250cbc7c4f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6680c63875d4f82a8a9dbe5ecaeb83b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:34.461664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b86aa-df16-4058-a399-d250cbc7c4f9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:34.509015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e15b86aa-df16-4058-a399-d250cbc7c4f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6680c63875d4f82a8a9dbe5ecaeb83b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:34.521617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e15b86aa-df16-4058-a399-d250cbc7c4f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6680c63875d4f82a8a9dbe5ecaeb83b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:34.522303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e15b86aa-df16-4058-a399-d250cbc7c4f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6680c63875d4f82a8a9dbe5ecaeb83b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:34.522904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e15b86aa-df16-4058-a399-d250cbc7c4f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6680c63875d4f82a8a9dbe5ecaeb83b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:34.529976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e15b86aa-df16-4058-a399-d250cbc7c4f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6680c63875d4f82a8a9dbe5ecaeb83b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:34.542432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1099/2234] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:32:34 2022] DELETE /identity/v3/users/b6680c63875d4f82a8a9dbe5ecaeb83b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:34.546987 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd06bf-81a7-4158-8e3d-e6053e96a564 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:34.547462 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd06bf-81a7-4158-8e3d-e6053e96a564 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:34.547587 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd06bf-81a7-4158-8e3d-e6053e96a564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:34.565938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-e5bd06bf-81a7-4158-8e3d-e6053e96a564 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:34.567935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e5bd06bf-81a7-4158-8e3d-e6053e96a564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:34.612593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:34.634142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1136/2235] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:34.643795 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b86aa-df16-4058-a399-d250cbc7c4f9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:34.645218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e15b86aa-df16-4058-a399-d250cbc7c4f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:34.660531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:34.707771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b86aa-df16-4058-a399-d250cbc7c4f9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:34.711484 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1906204-281c-47ec-89f0-a6508ec39e2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:34.712143 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1906204-281c-47ec-89f0-a6508ec39e2f service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:34.712425 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1906204-281c-47ec-89f0-a6508ec39e2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:34.712701 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1906204-281c-47ec-89f0-a6508ec39e2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:34.722288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1906204-281c-47ec-89f0-a6508ec39e2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:34.723881 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1906204-281c-47ec-89f0-a6508ec39e2f service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:34.733357 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1100/2236] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:32:35.032949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd06bf-81a7-4158-8e3d-e6053e96a564 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:35.034415 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e5bd06bf-81a7-4158-8e3d-e6053e96a564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:35.048133 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd06bf-81a7-4158-8e3d-e6053e96a564 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:35.053452 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665b2fd-0996-43ff-b261-44282885f09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:35.054329 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665b2fd-0996-43ff-b261-44282885f09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:35.054620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665b2fd-0996-43ff-b261-44282885f09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:35.054899 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665b2fd-0996-43ff-b261-44282885f09b admin admin] PATH_INFO: `/v3/projects/fe9f05b7338d4b5d9c7478f9b542076f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:35.056106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665b2fd-0996-43ff-b261-44282885f09b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe9f05b7338d4b5d9c7478f9b542076f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:35.058176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665b2fd-0996-43ff-b261-44282885f09b admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:35.111152 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9665b2fd-0996-43ff-b261-44282885f09b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe9f05b7338d4b5d9c7478f9b542076f', 'request_id': 'req-9665b2fd-0996-43ff-b261-44282885f09b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:35.117178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9665b2fd-0996-43ff-b261-44282885f09b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe9f05b7338d4b5d9c7478f9b542076f', 'request_id': 'req-9665b2fd-0996-43ff-b261-44282885f09b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:35.123308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9665b2fd-0996-43ff-b261-44282885f09b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe9f05b7338d4b5d9c7478f9b542076f', 'request_id': 'req-9665b2fd-0996-43ff-b261-44282885f09b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:35.135390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9665b2fd-0996-43ff-b261-44282885f09b admin admin] The token cache is being invalidate because project fe9f05b7338d4b5d9c7478f9b542076f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:32:35.135672 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9665b2fd-0996-43ff-b261-44282885f09b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:35.136284 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9665b2fd-0996-43ff-b261-44282885f09b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:35.137928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1137/2237] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:32:35 2022] DELETE /identity/v3/projects/fe9f05b7338d4b5d9c7478f9b542076f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:35.141935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1906204-281c-47ec-89f0-a6508ec39e2f service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:35.152492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1906204-281c-47ec-89f0-a6508ec39e2f service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:35.212968 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e463a4-60e7-427d-a46d-5818065f5022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:35.213877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e463a4-60e7-427d-a46d-5818065f5022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:35.214250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e463a4-60e7-427d-a46d-5818065f5022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:35.214415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e463a4-60e7-427d-a46d-5818065f5022 admin admin] PATH_INFO: `/v3/users/023e2d492dca46f8b11c9e656a6170ce` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:35.215604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e463a4-60e7-427d-a46d-5818065f5022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=023e2d492dca46f8b11c9e656a6170ce)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:35.217610 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e463a4-60e7-427d-a46d-5818065f5022 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:35.275662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-39e463a4-60e7-427d-a46d-5818065f5022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '023e2d492dca46f8b11c9e656a6170ce'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:35.289263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-39e463a4-60e7-427d-a46d-5818065f5022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '023e2d492dca46f8b11c9e656a6170ce'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:35.290884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-39e463a4-60e7-427d-a46d-5818065f5022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '023e2d492dca46f8b11c9e656a6170ce'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:35.291666 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-39e463a4-60e7-427d-a46d-5818065f5022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '023e2d492dca46f8b11c9e656a6170ce'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:35.299985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-39e463a4-60e7-427d-a46d-5818065f5022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '023e2d492dca46f8b11c9e656a6170ce'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:35.314805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1101/2238] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:32:35 2022] DELETE /identity/v3/users/023e2d492dca46f8b11c9e656a6170ce => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:35.612467 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665b2fd-0996-43ff-b261-44282885f09b admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:35.613984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9665b2fd-0996-43ff-b261-44282885f09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:35.628801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665b2fd-0996-43ff-b261-44282885f09b admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:35.690478 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb52c8f-0ab1-4835-a172-7ab3354c3635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:35.691260 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb52c8f-0ab1-4835-a172-7ab3354c3635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:35.692048 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb52c8f-0ab1-4835-a172-7ab3354c3635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:35.692321 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb52c8f-0ab1-4835-a172-7ab3354c3635 admin admin] PATH_INFO: `/v3/projects/d765911ee6174e1fb8b795a7e2dd46bd` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:35.693726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb52c8f-0ab1-4835-a172-7ab3354c3635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d765911ee6174e1fb8b795a7e2dd46bd)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:35.695705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb52c8f-0ab1-4835-a172-7ab3354c3635 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:35.745823 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9cb52c8f-0ab1-4835-a172-7ab3354c3635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd765911ee6174e1fb8b795a7e2dd46bd', 'request_id': 'req-9cb52c8f-0ab1-4835-a172-7ab3354c3635'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:35.751891 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9cb52c8f-0ab1-4835-a172-7ab3354c3635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd765911ee6174e1fb8b795a7e2dd46bd', 'request_id': 'req-9cb52c8f-0ab1-4835-a172-7ab3354c3635'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:35.759227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9cb52c8f-0ab1-4835-a172-7ab3354c3635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd765911ee6174e1fb8b795a7e2dd46bd', 'request_id': 'req-9cb52c8f-0ab1-4835-a172-7ab3354c3635'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:35.767879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:35.774760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9cb52c8f-0ab1-4835-a172-7ab3354c3635 admin admin] The token cache is being invalidate because project d765911ee6174e1fb8b795a7e2dd46bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:32:35.775374 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9cb52c8f-0ab1-4835-a172-7ab3354c3635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:35.776109 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9cb52c8f-0ab1-4835-a172-7ab3354c3635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:35.777816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1138/2239] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:32:35 2022] DELETE /identity/v3/projects/d765911ee6174e1fb8b795a7e2dd46bd => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:35.782689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dd797-50cd-4cc3-97ba-4c10aec9b289 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:35.783172 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dd797-50cd-4cc3-97ba-4c10aec9b289 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:35.783272 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dd797-50cd-4cc3-97ba-4c10aec9b289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:35.802752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8b9dd797-50cd-4cc3-97ba-4c10aec9b289 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:35.804035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8b9dd797-50cd-4cc3-97ba-4c10aec9b289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:35.835753 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:35.869262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1102/2240] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:35.878178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf23c72-4a10-4867-9589-29a71cf9b0e6 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:35.878465 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf23c72-4a10-4867-9589-29a71cf9b0e6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:35.878742 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf23c72-4a10-4867-9589-29a71cf9b0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:35.897453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-bcf23c72-4a10-4867-9589-29a71cf9b0e6 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:35.898934 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-bcf23c72-4a10-4867-9589-29a71cf9b0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:35.906929 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1139/2241] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:35.911871 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9dd797-50cd-4cc3-97ba-4c10aec9b289 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:35.921811 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9dd797-50cd-4cc3-97ba-4c10aec9b289 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:35.926570 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb82ff1-3ff6-4dff-857c-54ba4de15ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:35.927224 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb82ff1-3ff6-4dff-857c-54ba4de15ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:35.927508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb82ff1-3ff6-4dff-857c-54ba4de15ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:35.927788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb82ff1-3ff6-4dff-857c-54ba4de15ee1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:35.928525 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb82ff1-3ff6-4dff-857c-54ba4de15ee1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:32:35.928834 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb82ff1-3ff6-4dff-857c-54ba4de15ee1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:35.930486 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb82ff1-3ff6-4dff-857c-54ba4de15ee1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:35.943022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1103/2242] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:32:35.947568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf23c72-4a10-4867-9589-29a71cf9b0e6 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:35.957423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf23c72-4a10-4867-9589-29a71cf9b0e6 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:35.962359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50492c-1250-4128-8b20-a2fff5a606cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:35.963008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50492c-1250-4128-8b20-a2fff5a606cc admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:35.963312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50492c-1250-4128-8b20-a2fff5a606cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:35.963586 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50492c-1250-4128-8b20-a2fff5a606cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:35.964396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50492c-1250-4128-8b20-a2fff5a606cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:35.966313 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50492c-1250-4128-8b20-a2fff5a606cc admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:35.997088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1140/2243] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:32:35 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:32:35.998253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb82ff1-3ff6-4dff-857c-54ba4de15ee1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:36.009530 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb82ff1-3ff6-4dff-857c-54ba4de15ee1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:36.078254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97011ca2-164e-4f88-9314-2764e2bfb7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:36.078935 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97011ca2-164e-4f88-9314-2764e2bfb7ab admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:36.079220 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97011ca2-164e-4f88-9314-2764e2bfb7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:36.079495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97011ca2-164e-4f88-9314-2764e2bfb7ab admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:36.080186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97011ca2-164e-4f88-9314-2764e2bfb7ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:36.082660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97011ca2-164e-4f88-9314-2764e2bfb7ab admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:36.104582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1104/2244] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:32:35 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:32:36.108699 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50492c-1250-4128-8b20-a2fff5a606cc admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:36.119435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50492c-1250-4128-8b20-a2fff5a606cc admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:36.124174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb15fd-be54-4be4-bd46-40d108044f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:36.124838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb15fd-be54-4be4-bd46-40d108044f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:36.125118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb15fd-be54-4be4-bd46-40d108044f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:36.125417 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb15fd-be54-4be4-bd46-40d108044f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:36.126177 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb15fd-be54-4be4-bd46-40d108044f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:36.127977 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb15fd-be54-4be4-bd46-40d108044f27 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:36.139971 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1141/2245] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:32:36.144105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97011ca2-164e-4f88-9314-2764e2bfb7ab admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:36.155542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97011ca2-164e-4f88-9314-2764e2bfb7ab admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:36.162014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87c55e-c774-46d0-a3f3-e53517a496d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:36.162732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87c55e-c774-46d0-a3f3-e53517a496d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:36.163019 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87c55e-c774-46d0-a3f3-e53517a496d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:36.163298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87c55e-c774-46d0-a3f3-e53517a496d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:36.164050 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87c55e-c774-46d0-a3f3-e53517a496d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:36.165689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87c55e-c774-46d0-a3f3-e53517a496d4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:36.176234 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1105/2246] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:32:36.180743 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb15fd-be54-4be4-bd46-40d108044f27 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:36.191484 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb15fd-be54-4be4-bd46-40d108044f27 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:36.196570 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c034f-73b8-426d-9b6d-4486d92bc36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:36.197617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c034f-73b8-426d-9b6d-4486d92bc36a admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:36.201723 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c034f-73b8-426d-9b6d-4486d92bc36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:36.202024 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c034f-73b8-426d-9b6d-4486d92bc36a admin admin] PATH_INFO: `/v3/projects/d4879b5ec91c44bfb18788d2b3b8462b/users/d5778d727f72437d96614034afa4933a/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:36.218745 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c034f-73b8-426d-9b6d-4486d92bc36a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4879b5ec91c44bfb18788d2b3b8462b, user_id=d5778d727f72437d96614034afa4933a, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:36.221487 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c034f-73b8-426d-9b6d-4486d92bc36a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:36.234643 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1142/2247] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:32:36 2022] PUT /identity/v3/projects/d4879b5ec91c44bfb18788d2b3b8462b/users/d5778d727f72437d96614034afa4933a/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:36.248924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0cf08-5d8a-4701-81c3-4ef433b3d7aa None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:36.248924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0cf08-5d8a-4701-81c3-4ef433b3d7aa None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:36.248924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0cf08-5d8a-4701-81c3-4ef433b3d7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:36.275809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2db0cf08-5d8a-4701-81c3-4ef433b3d7aa None None] MFA Rules not processed for user `d5778d727f72437d96614034afa4933a`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:36.277152 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2db0cf08-5d8a-4701-81c3-4ef433b3d7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:36.352933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1106/2248] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:36.367157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c034f-73b8-426d-9b6d-4486d92bc36a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:36.369160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2a9c034f-73b8-426d-9b6d-4486d92bc36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:36.455268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c034f-73b8-426d-9b6d-4486d92bc36a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:36.459766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a6585-8746-483c-be88-4f18c37aee53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:36.460505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a6585-8746-483c-be88-4f18c37aee53 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:36.460857 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a6585-8746-483c-be88-4f18c37aee53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:36.461241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a6585-8746-483c-be88-4f18c37aee53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:36.474163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a6585-8746-483c-be88-4f18c37aee53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:36.475228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a6585-8746-483c-be88-4f18c37aee53 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:36.490455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1143/2249] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:32:36.517008 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0cf08-5d8a-4701-81c3-4ef433b3d7aa None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:36.552504 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:36.606482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0cf08-5d8a-4701-81c3-4ef433b3d7aa None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:36.611137 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232281f3-ab32-4085-a6f9-09e893dd4949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:36.612155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232281f3-ab32-4085-a6f9-09e893dd4949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:36.612429 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232281f3-ab32-4085-a6f9-09e893dd4949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:36.612724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232281f3-ab32-4085-a6f9-09e893dd4949 admin admin] PATH_INFO: `/v3/users/d5778d727f72437d96614034afa4933a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:36.616068 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232281f3-ab32-4085-a6f9-09e893dd4949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5778d727f72437d96614034afa4933a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:36.618181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232281f3-ab32-4085-a6f9-09e893dd4949 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:36.672290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-232281f3-ab32-4085-a6f9-09e893dd4949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5778d727f72437d96614034afa4933a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:36.685428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-232281f3-ab32-4085-a6f9-09e893dd4949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5778d727f72437d96614034afa4933a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:36.686112 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-232281f3-ab32-4085-a6f9-09e893dd4949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5778d727f72437d96614034afa4933a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:36.686860 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-232281f3-ab32-4085-a6f9-09e893dd4949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5778d727f72437d96614034afa4933a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:36.693785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-232281f3-ab32-4085-a6f9-09e893dd4949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5778d727f72437d96614034afa4933a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:36.712431 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1107/2250] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:32:36 2022] DELETE /identity/v3/users/d5778d727f72437d96614034afa4933a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:36.717710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e656c-92dc-437e-82b0-1a2fd7704e2b None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:36.718503 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e656c-92dc-437e-82b0-1a2fd7704e2b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:36.718971 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e656c-92dc-437e-82b0-1a2fd7704e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:36.737756 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-2f1e656c-92dc-437e-82b0-1a2fd7704e2b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:36.740508 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2f1e656c-92dc-437e-82b0-1a2fd7704e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:36.804481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1144/2251] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:36.815061 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232281f3-ab32-4085-a6f9-09e893dd4949 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:36.815821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-232281f3-ab32-4085-a6f9-09e893dd4949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:36.876503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232281f3-ab32-4085-a6f9-09e893dd4949 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:36.880489 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69025bff-1ff5-4cd3-9514-808f287c8cc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:36.881167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69025bff-1ff5-4cd3-9514-808f287c8cc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:36.881463 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69025bff-1ff5-4cd3-9514-808f287c8cc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:36.881747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69025bff-1ff5-4cd3-9514-808f287c8cc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:36.891584 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69025bff-1ff5-4cd3-9514-808f287c8cc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:36.893036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69025bff-1ff5-4cd3-9514-808f287c8cc3 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:36.903474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1108/2252] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:32:37.430440 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e656c-92dc-437e-82b0-1a2fd7704e2b None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:37.432221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2f1e656c-92dc-437e-82b0-1a2fd7704e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:37.447416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e656c-92dc-437e-82b0-1a2fd7704e2b None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:37.454200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804365a-87ad-4a23-907f-4cf11bf4b603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:37.455546 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804365a-87ad-4a23-907f-4cf11bf4b603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:37.455776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804365a-87ad-4a23-907f-4cf11bf4b603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:37.456054 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804365a-87ad-4a23-907f-4cf11bf4b603 admin admin] PATH_INFO: `/v3/projects/d4879b5ec91c44bfb18788d2b3b8462b` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:37.459329 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804365a-87ad-4a23-907f-4cf11bf4b603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4879b5ec91c44bfb18788d2b3b8462b)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:37.461898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804365a-87ad-4a23-907f-4cf11bf4b603 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:37.499067 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:37.513587 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8804365a-87ad-4a23-907f-4cf11bf4b603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4879b5ec91c44bfb18788d2b3b8462b', 'request_id': 'req-8804365a-87ad-4a23-907f-4cf11bf4b603'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:37.520710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8804365a-87ad-4a23-907f-4cf11bf4b603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4879b5ec91c44bfb18788d2b3b8462b', 'request_id': 'req-8804365a-87ad-4a23-907f-4cf11bf4b603'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:37.527388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8804365a-87ad-4a23-907f-4cf11bf4b603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4879b5ec91c44bfb18788d2b3b8462b', 'request_id': 'req-8804365a-87ad-4a23-907f-4cf11bf4b603'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:37.540285 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8804365a-87ad-4a23-907f-4cf11bf4b603 admin admin] The token cache is being invalidate because project d4879b5ec91c44bfb18788d2b3b8462b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:32:37.540609 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8804365a-87ad-4a23-907f-4cf11bf4b603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:37.541268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8804365a-87ad-4a23-907f-4cf11bf4b603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:37.544461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1145/2253] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:32:37 2022] DELETE /identity/v3/projects/d4879b5ec91c44bfb18788d2b3b8462b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:37.549510 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd6c11-819d-48f2-a721-e77d02500f62 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:37.549800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd6c11-819d-48f2-a721-e77d02500f62 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:37.550075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd6c11-819d-48f2-a721-e77d02500f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:37.569912 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-61cd6c11-819d-48f2-a721-e77d02500f62 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:37.571151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-61cd6c11-819d-48f2-a721-e77d02500f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:37.618783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:37.638328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1109/2254] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:37.648680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9bb25-2e4d-4d86-b88e-41f309eac741 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:37.649195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9bb25-2e4d-4d86-b88e-41f309eac741 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:37.649730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9bb25-2e4d-4d86-b88e-41f309eac741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:37.672612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-96d9bb25-2e4d-4d86-b88e-41f309eac741 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:37.673902 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-96d9bb25-2e4d-4d86-b88e-41f309eac741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:37.688234 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1146/2255] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:37.692606 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd6c11-819d-48f2-a721-e77d02500f62 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:37.703245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd6c11-819d-48f2-a721-e77d02500f62 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:37.708647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cecef4-cc77-4adb-8785-e699e366ad5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:37.709688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cecef4-cc77-4adb-8785-e699e366ad5a admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:37.709688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cecef4-cc77-4adb-8785-e699e366ad5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:37.709894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cecef4-cc77-4adb-8785-e699e366ad5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:37.710633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cecef4-cc77-4adb-8785-e699e366ad5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:32:37.711074 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cecef4-cc77-4adb-8785-e699e366ad5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:37.712777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cecef4-cc77-4adb-8785-e699e366ad5a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:37.731214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1110/2256] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:32:37.732555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9bb25-2e4d-4d86-b88e-41f309eac741 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:37.742864 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9bb25-2e4d-4d86-b88e-41f309eac741 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:37.748798 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad899613-4f1d-4d08-951d-e09a4babec56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:37.748798 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad899613-4f1d-4d08-951d-e09a4babec56 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:37.749046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad899613-4f1d-4d08-951d-e09a4babec56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:37.749365 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad899613-4f1d-4d08-951d-e09a4babec56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:37.750634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad899613-4f1d-4d08-951d-e09a4babec56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:37.755633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad899613-4f1d-4d08-951d-e09a4babec56 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:37.786177 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1147/2257] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:32:37 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:32:37.790487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cecef4-cc77-4adb-8785-e699e366ad5a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:37.802475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cecef4-cc77-4adb-8785-e699e366ad5a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:37.869398 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16b3b8-69c1-4e1e-b038-c4e8b65386f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:37.870080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16b3b8-69c1-4e1e-b038-c4e8b65386f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:37.870370 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16b3b8-69c1-4e1e-b038-c4e8b65386f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:37.870651 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16b3b8-69c1-4e1e-b038-c4e8b65386f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:37.871346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16b3b8-69c1-4e1e-b038-c4e8b65386f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:37.873584 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16b3b8-69c1-4e1e-b038-c4e8b65386f5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:37.894462 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1111/2258] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:32:37 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:32:37.899236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad899613-4f1d-4d08-951d-e09a4babec56 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:37.910896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad899613-4f1d-4d08-951d-e09a4babec56 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:37.915808 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe29a4-5ada-4822-9844-fd11d5589fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:37.916464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe29a4-5ada-4822-9844-fd11d5589fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:37.916756 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe29a4-5ada-4822-9844-fd11d5589fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:37.917035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe29a4-5ada-4822-9844-fd11d5589fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:37.917804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe29a4-5ada-4822-9844-fd11d5589fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:37.919409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe29a4-5ada-4822-9844-fd11d5589fb6 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:37.931686 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1148/2259] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:32:37.935674 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16b3b8-69c1-4e1e-b038-c4e8b65386f5 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:37.946169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16b3b8-69c1-4e1e-b038-c4e8b65386f5 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:37.950820 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83472511-1370-4986-ae5c-78f4a9791aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:37.951572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83472511-1370-4986-ae5c-78f4a9791aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:37.951856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83472511-1370-4986-ae5c-78f4a9791aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:37.952132 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83472511-1370-4986-ae5c-78f4a9791aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:37.952948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83472511-1370-4986-ae5c-78f4a9791aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:37.954622 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83472511-1370-4986-ae5c-78f4a9791aa8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:37.964997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1112/2260] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:32:37.969265 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe29a4-5ada-4822-9844-fd11d5589fb6 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:37.980574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe29a4-5ada-4822-9844-fd11d5589fb6 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:37.985581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11fd4f-6575-4eb3-91c3-f6bd82788805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:37.986461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11fd4f-6575-4eb3-91c3-f6bd82788805 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:37.986741 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11fd4f-6575-4eb3-91c3-f6bd82788805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:37.987016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11fd4f-6575-4eb3-91c3-f6bd82788805 admin admin] PATH_INFO: `/v3/projects/d8ba7f503b3c4afeb9736b241ac2bc98/users/51afc55d8978440bb5c9e951f9c51c59/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:38.002408 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11fd4f-6575-4eb3-91c3-f6bd82788805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8ba7f503b3c4afeb9736b241ac2bc98, user_id=51afc55d8978440bb5c9e951f9c51c59, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:38.005266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11fd4f-6575-4eb3-91c3-f6bd82788805 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:38.019243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1149/2261] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:32:37 2022] PUT /identity/v3/projects/d8ba7f503b3c4afeb9736b241ac2bc98/users/51afc55d8978440bb5c9e951f9c51c59/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:32:38.025326 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d98ab-3372-43a9-8f2b-8da1d3ce6b28 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:38.025755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d98ab-3372-43a9-8f2b-8da1d3ce6b28 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:38.026186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d98ab-3372-43a9-8f2b-8da1d3ce6b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:38.045668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ea9d98ab-3372-43a9-8f2b-8da1d3ce6b28 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:38.047167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ea9d98ab-3372-43a9-8f2b-8da1d3ce6b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:38.108629 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1113/2262] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:32:38.119299 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11fd4f-6575-4eb3-91c3-f6bd82788805 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:38.121225 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0c11fd4f-6575-4eb3-91c3-f6bd82788805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:38.180188 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11fd4f-6575-4eb3-91c3-f6bd82788805 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:38.183886 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9322b67-adfd-49a6-859c-3a3f52eeb724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:38.184543 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9322b67-adfd-49a6-859c-3a3f52eeb724 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:38.184828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9322b67-adfd-49a6-859c-3a3f52eeb724 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:38.185109 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9322b67-adfd-49a6-859c-3a3f52eeb724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:38.195864 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9322b67-adfd-49a6-859c-3a3f52eeb724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:38.197430 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9322b67-adfd-49a6-859c-3a3f52eeb724 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:38.210588 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1150/2263] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:32:39.297517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac615e2e-8374-4ed3-bfe2-c88e1339862e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:39.297981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac615e2e-8374-4ed3-bfe2-c88e1339862e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:39.298086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac615e2e-8374-4ed3-bfe2-c88e1339862e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:39.339722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ac615e2e-8374-4ed3-bfe2-c88e1339862e None None] MFA Rules not processed for user `51afc55d8978440bb5c9e951f9c51c59`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:39.340933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ac615e2e-8374-4ed3-bfe2-c88e1339862e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:39.397517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:39.432143 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1114/2264] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:32:39.444513 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9322b67-adfd-49a6-859c-3a3f52eeb724 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:39.446265 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e9322b67-adfd-49a6-859c-3a3f52eeb724 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:39.515090 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9322b67-adfd-49a6-859c-3a3f52eeb724 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:39.520008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b8322-aa4b-45e8-ae1c-b45a9bedd53b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:39.520666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b8322-aa4b-45e8-ae1c-b45a9bedd53b service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:39.521039 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b8322-aa4b-45e8-ae1c-b45a9bedd53b service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:39.521418 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b8322-aa4b-45e8-ae1c-b45a9bedd53b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:39.531327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b8322-aa4b-45e8-ae1c-b45a9bedd53b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:39.532855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b8322-aa4b-45e8-ae1c-b45a9bedd53b service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:39.545310 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1151/2265] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:32:41.008036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7303018-544d-4ce8-833b-d54eaa3c9480 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:41.008532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7303018-544d-4ce8-833b-d54eaa3c9480 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:41.008625 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7303018-544d-4ce8-833b-d54eaa3c9480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:41.027495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e7303018-544d-4ce8-833b-d54eaa3c9480 None None] MFA Rules not processed for user `51afc55d8978440bb5c9e951f9c51c59`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:41.028844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e7303018-544d-4ce8-833b-d54eaa3c9480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:41.037191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1115/2266] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:49.538485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:49.983623 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d18294-d270-48e1-a37f-548446bece60 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:49.983916 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d18294-d270-48e1-a37f-548446bece60 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:49.984202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d18294-d270-48e1-a37f-548446bece60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:50.002312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-c9d18294-d270-48e1-a37f-548446bece60 None None] MFA Rules not processed for user `5507329a6bb84cfbbdac99b85f346ab8`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:50.003529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c9d18294-d270-48e1-a37f-548446bece60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:50.073814 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1152/2267] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:51.021158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=9 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:55.294892 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7303018-544d-4ce8-833b-d54eaa3c9480 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:55.297031 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e7303018-544d-4ce8-833b-d54eaa3c9480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:55.312514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7303018-544d-4ce8-833b-d54eaa3c9480 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:55.317891 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbb822-7d31-4bd6-a588-8beb4b5f183b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:55.318964 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbb822-7d31-4bd6-a588-8beb4b5f183b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:55.319405 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbb822-7d31-4bd6-a588-8beb4b5f183b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:55.319830 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbb822-7d31-4bd6-a588-8beb4b5f183b admin admin] PATH_INFO: `/v3/users/364ee8b6b75d495ca3dc60916daa0afc` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:55.321212 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbb822-7d31-4bd6-a588-8beb4b5f183b admin admin] RBAC: Authorizing `identity:delete_user(user_id=364ee8b6b75d495ca3dc60916daa0afc)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:55.323309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbb822-7d31-4bd6-a588-8beb4b5f183b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:55.375461 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a9cbb822-7d31-4bd6-a588-8beb4b5f183b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '364ee8b6b75d495ca3dc60916daa0afc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:55.398592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a9cbb822-7d31-4bd6-a588-8beb4b5f183b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '364ee8b6b75d495ca3dc60916daa0afc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:55.399339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a9cbb822-7d31-4bd6-a588-8beb4b5f183b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '364ee8b6b75d495ca3dc60916daa0afc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:55.399959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a9cbb822-7d31-4bd6-a588-8beb4b5f183b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '364ee8b6b75d495ca3dc60916daa0afc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:55.409923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a9cbb822-7d31-4bd6-a588-8beb4b5f183b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '364ee8b6b75d495ca3dc60916daa0afc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:55.431390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1116/2268] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:32:55 2022] DELETE /identity/v3/users/364ee8b6b75d495ca3dc60916daa0afc => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:55.434960 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac132c9f-b104-4076-a818-af94d588f56e None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:55.435479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac132c9f-b104-4076-a818-af94d588f56e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:55.435567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac132c9f-b104-4076-a818-af94d588f56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:55.453719 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ac132c9f-b104-4076-a818-af94d588f56e None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:55.454947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ac132c9f-b104-4076-a818-af94d588f56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:55.535638 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1153/2269] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:55.563717 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbb822-7d31-4bd6-a588-8beb4b5f183b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:55.565302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a9cbb822-7d31-4bd6-a588-8beb4b5f183b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:55.638554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbb822-7d31-4bd6-a588-8beb4b5f183b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:55.642487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5318fa-4e05-413d-83e4-0042e298223c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:55.643166 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5318fa-4e05-413d-83e4-0042e298223c service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:55.643510 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5318fa-4e05-413d-83e4-0042e298223c service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:55.643832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5318fa-4e05-413d-83e4-0042e298223c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:55.655227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5318fa-4e05-413d-83e4-0042e298223c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:55.656732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5318fa-4e05-413d-83e4-0042e298223c service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:55.667917 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1117/2270] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:32:56.080452 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac132c9f-b104-4076-a818-af94d588f56e None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:56.082615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ac132c9f-b104-4076-a818-af94d588f56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:56.100311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac132c9f-b104-4076-a818-af94d588f56e None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:56.107246 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c1bfd-182a-478f-9c5d-45fea4de5dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:56.108335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c1bfd-182a-478f-9c5d-45fea4de5dcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:56.108624 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c1bfd-182a-478f-9c5d-45fea4de5dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:56.108900 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c1bfd-182a-478f-9c5d-45fea4de5dcf admin admin] PATH_INFO: `/v3/projects/9425e51c3abd4c8dbc9a6de5c2f438c7` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:56.110189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c1bfd-182a-478f-9c5d-45fea4de5dcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9425e51c3abd4c8dbc9a6de5c2f438c7)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:56.113257 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c1bfd-182a-478f-9c5d-45fea4de5dcf admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:56.164313 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1a3c1bfd-182a-478f-9c5d-45fea4de5dcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9425e51c3abd4c8dbc9a6de5c2f438c7', 'request_id': 'req-1a3c1bfd-182a-478f-9c5d-45fea4de5dcf'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:56.169924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1a3c1bfd-182a-478f-9c5d-45fea4de5dcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9425e51c3abd4c8dbc9a6de5c2f438c7', 'request_id': 'req-1a3c1bfd-182a-478f-9c5d-45fea4de5dcf'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:56.177186 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1a3c1bfd-182a-478f-9c5d-45fea4de5dcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9425e51c3abd4c8dbc9a6de5c2f438c7', 'request_id': 'req-1a3c1bfd-182a-478f-9c5d-45fea4de5dcf'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:56.190556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1a3c1bfd-182a-478f-9c5d-45fea4de5dcf admin admin] The token cache is being invalidate because project 9425e51c3abd4c8dbc9a6de5c2f438c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:32:56.191045 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1a3c1bfd-182a-478f-9c5d-45fea4de5dcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:56.191591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1a3c1bfd-182a-478f-9c5d-45fea4de5dcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:32:56.193227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1154/2271] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:32:56 2022] DELETE /identity/v3/projects/9425e51c3abd4c8dbc9a6de5c2f438c7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:56.198302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97965a1-bab0-46c8-b5be-b9c702e0e885 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:56.198737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97965a1-bab0-46c8-b5be-b9c702e0e885 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:56.199007 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97965a1-bab0-46c8-b5be-b9c702e0e885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:56.217002 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b97965a1-bab0-46c8-b5be-b9c702e0e885 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:56.218228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b97965a1-bab0-46c8-b5be-b9c702e0e885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:56.283173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1118/2272] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:32:56.291050 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9812a-7000-4d78-98c1-6482e728b178 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:56.291339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9812a-7000-4d78-98c1-6482e728b178 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:56.291622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9812a-7000-4d78-98c1-6482e728b178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:56.310821 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-e9a9812a-7000-4d78-98c1-6482e728b178 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:56.312043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e9a9812a-7000-4d78-98c1-6482e728b178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:56.321844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1155/2273] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:32:56.326119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97965a1-bab0-46c8-b5be-b9c702e0e885 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:56.336532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97965a1-bab0-46c8-b5be-b9c702e0e885 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:56.341121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c23173-b15a-4771-b639-c47fc39ee4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:56.341805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c23173-b15a-4771-b639-c47fc39ee4eb admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:56.342088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c23173-b15a-4771-b639-c47fc39ee4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:56.342366 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c23173-b15a-4771-b639-c47fc39ee4eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:56.343097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c23173-b15a-4771-b639-c47fc39ee4eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:32:56.343404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c23173-b15a-4771-b639-c47fc39ee4eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:56.345001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c23173-b15a-4771-b639-c47fc39ee4eb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:56.357086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1119/2274] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:32:56.361324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9812a-7000-4d78-98c1-6482e728b178 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:56.370680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9812a-7000-4d78-98c1-6482e728b178 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:56.375368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f8426-ece7-4fcb-8bce-c5e08382d823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:56.376014 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f8426-ece7-4fcb-8bce-c5e08382d823 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:56.376298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f8426-ece7-4fcb-8bce-c5e08382d823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:56.376574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f8426-ece7-4fcb-8bce-c5e08382d823 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:56.377308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f8426-ece7-4fcb-8bce-c5e08382d823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:56.379489 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f8426-ece7-4fcb-8bce-c5e08382d823 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:56.399743 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:56.406607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1156/2275] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:32:56 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:32:56.411382 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c23173-b15a-4771-b639-c47fc39ee4eb admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:56.416617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:32:56.422668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c23173-b15a-4771-b639-c47fc39ee4eb admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:56.479052 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a3e93-6d28-4407-ad38-ae2dfd0b37de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:56.479769 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a3e93-6d28-4407-ad38-ae2dfd0b37de admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:56.480054 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a3e93-6d28-4407-ad38-ae2dfd0b37de admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:56.480331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a3e93-6d28-4407-ad38-ae2dfd0b37de admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:56.481016 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a3e93-6d28-4407-ad38-ae2dfd0b37de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:56.482971 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a3e93-6d28-4407-ad38-ae2dfd0b37de admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:56.501835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1120/2276] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:32:56 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:32:56.506032 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f8426-ece7-4fcb-8bce-c5e08382d823 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:56.516518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f8426-ece7-4fcb-8bce-c5e08382d823 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:56.521841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326fa14-dddf-4b2b-a704-5951f408897f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:56.522496 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326fa14-dddf-4b2b-a704-5951f408897f admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:56.522776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326fa14-dddf-4b2b-a704-5951f408897f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:56.523316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326fa14-dddf-4b2b-a704-5951f408897f admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:56.524076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326fa14-dddf-4b2b-a704-5951f408897f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:56.525717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326fa14-dddf-4b2b-a704-5951f408897f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:56.536644 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1157/2277] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:32:56.540337 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a3e93-6d28-4407-ad38-ae2dfd0b37de admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:56.550153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a3e93-6d28-4407-ad38-ae2dfd0b37de admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:56.554875 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b275679-44da-4222-a5e3-3558dda83145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:56.555526 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b275679-44da-4222-a5e3-3558dda83145 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:56.555812 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b275679-44da-4222-a5e3-3558dda83145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:56.556104 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b275679-44da-4222-a5e3-3558dda83145 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:56.556816 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b275679-44da-4222-a5e3-3558dda83145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:56.558439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b275679-44da-4222-a5e3-3558dda83145 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:56.568362 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1121/2278] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:32:56.572286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326fa14-dddf-4b2b-a704-5951f408897f admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:56.582761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326fa14-dddf-4b2b-a704-5951f408897f admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:56.587398 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4f917-d2c0-422c-8abb-4ee52076a25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:56.588280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4f917-d2c0-422c-8abb-4ee52076a25a admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:56.588551 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4f917-d2c0-422c-8abb-4ee52076a25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:56.588871 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4f917-d2c0-422c-8abb-4ee52076a25a admin admin] PATH_INFO: `/v3/projects/286e8f4ce0854bf3a8563bcea08ae93a/users/dee7cef53eb641bb93c92e3c524c59db/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:56.603781 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4f917-d2c0-422c-8abb-4ee52076a25a admin admin] RBAC: Authorizing `identity:create_grant(project_id=286e8f4ce0854bf3a8563bcea08ae93a, user_id=dee7cef53eb641bb93c92e3c524c59db, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:56.606579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4f917-d2c0-422c-8abb-4ee52076a25a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:56.618930 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1158/2279] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:32:56 2022] PUT /identity/v3/projects/286e8f4ce0854bf3a8563bcea08ae93a/users/dee7cef53eb641bb93c92e3c524c59db/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:32:56.623994 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14728dc-9766-4017-8c00-0f4f9de9b733 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:56.624287 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14728dc-9766-4017-8c00-0f4f9de9b733 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:56.624593 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14728dc-9766-4017-8c00-0f4f9de9b733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:56.642494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c14728dc-9766-4017-8c00-0f4f9de9b733 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:56.644256 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c14728dc-9766-4017-8c00-0f4f9de9b733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:56.704028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1122/2280] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:32:56.714194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4f917-d2c0-422c-8abb-4ee52076a25a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:32:56.716005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-03e4f917-d2c0-422c-8abb-4ee52076a25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:56.773878 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4f917-d2c0-422c-8abb-4ee52076a25a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:32:56.777554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95675fac-5d9a-4d7e-ad83-efc5bd171903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:32:56.778198 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95675fac-5d9a-4d7e-ad83-efc5bd171903 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:56.778487 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95675fac-5d9a-4d7e-ad83-efc5bd171903 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:56.779489 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95675fac-5d9a-4d7e-ad83-efc5bd171903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:56.791135 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95675fac-5d9a-4d7e-ad83-efc5bd171903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:32:56.792747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95675fac-5d9a-4d7e-ad83-efc5bd171903 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:32:56.803549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1159/2281] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:32:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:32:58.395400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27f6ad-03f0-48da-b75c-ccefa5d6b34a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:32:58.395862 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27f6ad-03f0-48da-b75c-ccefa5d6b34a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:32:58.396370 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27f6ad-03f0-48da-b75c-ccefa5d6b34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:32:58.416317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9e27f6ad-03f0-48da-b75c-ccefa5d6b34a None None] MFA Rules not processed for user `51afc55d8978440bb5c9e951f9c51c59`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:32:58.417590 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9e27f6ad-03f0-48da-b75c-ccefa5d6b34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:32:58.482797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1123/2282] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:32:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:33:01.623554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30de59-3547-4865-a890-886d63170b87 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:01.623990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30de59-3547-4865-a890-886d63170b87 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:01.624113 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30de59-3547-4865-a890-886d63170b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:01.684990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-3e30de59-3547-4865-a890-886d63170b87 None None] MFA Rules not processed for user `dee7cef53eb641bb93c92e3c524c59db`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:33:01.686962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3e30de59-3547-4865-a890-886d63170b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:01.792204 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1160/2283] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:33:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:33:01.793500 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27f6ad-03f0-48da-b75c-ccefa5d6b34a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:01.805760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27f6ad-03f0-48da-b75c-ccefa5d6b34a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:01.811940 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d7c19-094d-4714-a4ac-6775d7c42b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:01.812737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d7c19-094d-4714-a4ac-6775d7c42b94 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:01.813282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d7c19-094d-4714-a4ac-6775d7c42b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:01.813778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d7c19-094d-4714-a4ac-6775d7c42b94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:01.814614 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d7c19-094d-4714-a4ac-6775d7c42b94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:01.816681 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d7c19-094d-4714-a4ac-6775d7c42b94 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:01.845822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1124/2284] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:33:01 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:33:01.850421 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30de59-3547-4865-a890-886d63170b87 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:01.863019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30de59-3547-4865-a890-886d63170b87 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:01.926830 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:33:01.936821 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bb9b8-3087-4ee1-9cad-d16198788849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:01.937686 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bb9b8-3087-4ee1-9cad-d16198788849 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:01.938086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bb9b8-3087-4ee1-9cad-d16198788849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:01.938445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bb9b8-3087-4ee1-9cad-d16198788849 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:01.939250 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bb9b8-3087-4ee1-9cad-d16198788849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:01.942130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bb9b8-3087-4ee1-9cad-d16198788849 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:01.968982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1161/2285] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:33:01 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:33:01.970702 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d7c19-094d-4714-a4ac-6775d7c42b94 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:01.980119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:33:01.985221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d7c19-094d-4714-a4ac-6775d7c42b94 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:01.993322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04101b4-c9b5-4ca6-96c4-978a917c8343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:01.994022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04101b4-c9b5-4ca6-96c4-978a917c8343 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:01.994951 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04101b4-c9b5-4ca6-96c4-978a917c8343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:01.996029 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04101b4-c9b5-4ca6-96c4-978a917c8343 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:01.996852 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04101b4-c9b5-4ca6-96c4-978a917c8343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:01.998638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04101b4-c9b5-4ca6-96c4-978a917c8343 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:02.014623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1125/2286] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:33:02.016003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bb9b8-3087-4ee1-9cad-d16198788849 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:02.032922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bb9b8-3087-4ee1-9cad-d16198788849 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:02.041600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18358e0-0344-4e59-a832-12170588a062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:02.042816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18358e0-0344-4e59-a832-12170588a062 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:02.043584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18358e0-0344-4e59-a832-12170588a062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:02.043812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18358e0-0344-4e59-a832-12170588a062 admin admin] PATH_INFO: `/v3/projects/fe1038820b5b4754a05ac3bac4e9594e/users/1577a7a091eb466f9c083d084cc64b5c/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:02.067835 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18358e0-0344-4e59-a832-12170588a062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1038820b5b4754a05ac3bac4e9594e, user_id=1577a7a091eb466f9c083d084cc64b5c, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:02.071515 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18358e0-0344-4e59-a832-12170588a062 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:02.087582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1162/2287] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:33:02 2022] PUT /identity/v3/projects/fe1038820b5b4754a05ac3bac4e9594e/users/1577a7a091eb466f9c083d084cc64b5c/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:33:02.091920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04101b4-c9b5-4ca6-96c4-978a917c8343 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:02.103721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04101b4-c9b5-4ca6-96c4-978a917c8343 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:02.172629 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e02a31-481c-4b5f-90be-b2e8eacbe72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:02.173457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e02a31-481c-4b5f-90be-b2e8eacbe72e admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:02.173866 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e02a31-481c-4b5f-90be-b2e8eacbe72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:02.174227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e02a31-481c-4b5f-90be-b2e8eacbe72e admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:02.175117 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e02a31-481c-4b5f-90be-b2e8eacbe72e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:02.176855 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e02a31-481c-4b5f-90be-b2e8eacbe72e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:02.193969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1126/2288] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:33:02.194924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18358e0-0344-4e59-a832-12170588a062 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:02.208417 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18358e0-0344-4e59-a832-12170588a062 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:02.214544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234bb846-af1d-46e7-b0e3-b56cb6d62169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:02.215697 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bb846-af1d-46e7-b0e3-b56cb6d62169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:02.216081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bb846-af1d-46e7-b0e3-b56cb6d62169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:02.216708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bb846-af1d-46e7-b0e3-b56cb6d62169 admin admin] PATH_INFO: `/v3/projects/fe1038820b5b4754a05ac3bac4e9594e/users/1577a7a091eb466f9c083d084cc64b5c/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:02.218657 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234bb846-af1d-46e7-b0e3-b56cb6d62169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1038820b5b4754a05ac3bac4e9594e, user_id=1577a7a091eb466f9c083d084cc64b5c, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:02.223876 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234bb846-af1d-46e7-b0e3-b56cb6d62169 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:02.240533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1163/2289] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:33:02 2022] PUT /identity/v3/projects/fe1038820b5b4754a05ac3bac4e9594e/users/1577a7a091eb466f9c083d084cc64b5c/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:33:05.670406 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d18ea-4b93-43e4-a26e-1f83650b8642 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:05.670871 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d18ea-4b93-43e4-a26e-1f83650b8642 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:05.670871 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d18ea-4b93-43e4-a26e-1f83650b8642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:05.717048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6c4d18ea-4b93-43e4-a26e-1f83650b8642 None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:33:05.728773 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6c4d18ea-4b93-43e4-a26e-1f83650b8642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:05.897443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1127/2290] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:33:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:33:05.981800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234bb846-af1d-46e7-b0e3-b56cb6d62169 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:05.984230 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-234bb846-af1d-46e7-b0e3-b56cb6d62169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:06.121912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234bb846-af1d-46e7-b0e3-b56cb6d62169 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:06.128391 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e8306-f279-455e-9e99-1fb94607cf09 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:06.129245 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e8306-f279-455e-9e99-1fb94607cf09 service swift] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:06.129525 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e8306-f279-455e-9e99-1fb94607cf09 service swift] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:06.129794 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e8306-f279-455e-9e99-1fb94607cf09 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:06.139429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e8306-f279-455e-9e99-1fb94607cf09 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:06.141062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e8306-f279-455e-9e99-1fb94607cf09 service swift] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:06.151107 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1164/2291] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:33:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:33:09.535393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c57e7-0d0b-4c55-aed4-62d1676f6e30 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:09.535994 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c57e7-0d0b-4c55-aed4-62d1676f6e30 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:09.536186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c57e7-0d0b-4c55-aed4-62d1676f6e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:09.642564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bc1c57e7-0d0b-4c55-aed4-62d1676f6e30 None None] MFA Rules not processed for user `1577a7a091eb466f9c083d084cc64b5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:33:09.647119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bc1c57e7-0d0b-4c55-aed4-62d1676f6e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:09.796215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1128/2292] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:33:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:33:10.090993 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e8306-f279-455e-9e99-1fb94607cf09 service swift] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:10.092607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c23e8306-f279-455e-9e99-1fb94607cf09 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:10.299204 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e8306-f279-455e-9e99-1fb94607cf09 service swift] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:10.306900 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a7ce5-e242-4ff7-a425-a0a48c98c0ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:10.308443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a7ce5-e242-4ff7-a425-a0a48c98c0ac service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:10.309044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a7ce5-e242-4ff7-a425-a0a48c98c0ac service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:10.309562 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a7ce5-e242-4ff7-a425-a0a48c98c0ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:10.447885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:33:10.591862 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a7ce5-e242-4ff7-a425-a0a48c98c0ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:10.596537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a7ce5-e242-4ff7-a425-a0a48c98c0ac service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:10.634913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1165/2293] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:33:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:33:13.061595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b55d4-1c7a-48d1-b88b-e0f39976258e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:13.062100 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b55d4-1c7a-48d1-b88b-e0f39976258e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:13.062484 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b55d4-1c7a-48d1-b88b-e0f39976258e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:13.088119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ad6b55d4-1c7a-48d1-b88b-e0f39976258e None None] MFA Rules not processed for user `dee7cef53eb641bb93c92e3c524c59db`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:33:13.089385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ad6b55d4-1c7a-48d1-b88b-e0f39976258e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:13.099756 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1129/2294] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:33:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:33:18.866024 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a7ce5-e242-4ff7-a425-a0a48c98c0ac service nova] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:18.868460 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-4c3a7ce5-e242-4ff7-a425-a0a48c98c0ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:18.950541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a7ce5-e242-4ff7-a425-a0a48c98c0ac service nova] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:18.959319 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb77c7-a050-4f01-b94b-d51036431277 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:18.960355 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb77c7-a050-4f01-b94b-d51036431277 service placement] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:18.960644 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb77c7-a050-4f01-b94b-d51036431277 service placement] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:18.961184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb77c7-a050-4f01-b94b-d51036431277 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:18.963858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-85eb77c7-a050-4f01-b94b-d51036431277 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:18.976809 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eb77c7-a050-4f01-b94b-d51036431277 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:18.978777 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eb77c7-a050-4f01-b94b-d51036431277 service placement] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:18.991887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1166/2295] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:33:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:33:23.082119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:33:26.811605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0eb348-1479-4aca-9cdc-b1c0ae14b502 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:26.812148 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0eb348-1479-4aca-9cdc-b1c0ae14b502 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:26.812148 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0eb348-1479-4aca-9cdc-b1c0ae14b502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:26.829961 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-fb0eb348-1479-4aca-9cdc-b1c0ae14b502 None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:33:26.831147 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fb0eb348-1479-4aca-9cdc-b1c0ae14b502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:26.838259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1130/2296] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:33:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:33:26.850665 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb77c7-a050-4f01-b94b-d51036431277 service placement] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:26.862855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb77c7-a050-4f01-b94b-d51036431277 service placement] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:26.868328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec70f0-01d9-4f44-a143-a4573d831039 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:26.868964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec70f0-01d9-4f44-a143-a4573d831039 service swift] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:26.869277 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec70f0-01d9-4f44-a143-a4573d831039 service swift] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:26.869553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec70f0-01d9-4f44-a143-a4573d831039 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:26.877610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec70f0-01d9-4f44-a143-a4573d831039 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:26.879278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec70f0-01d9-4f44-a143-a4573d831039 service swift] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:26.887841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1167/2297] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:33:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:33:30.711270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a9edc-f2b1-4a42-acda-98a61fbf5a5f None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:30.711883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a9edc-f2b1-4a42-acda-98a61fbf5a5f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:30.711883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a9edc-f2b1-4a42-acda-98a61fbf5a5f None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:30.713234 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1131/2298] 10.222.0.42 () {54 vars in 921 bytes} [Tue Jul 19 15:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:33:30.719590 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc59d8a-5326-4d24-bbb6-24ff524bd55c None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:30.720286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc59d8a-5326-4d24-bbb6-24ff524bd55c None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:30.720899 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc59d8a-5326-4d24-bbb6-24ff524bd55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:30.743301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-bfc59d8a-5326-4d24-bbb6-24ff524bd55c None None] MFA Rules not processed for user `bff6a2d4bc734904b2615608a86b097d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:33:30.744942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-bfc59d8a-5326-4d24-bbb6-24ff524bd55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:30.756997 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1168/2299] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Jul 19 15:33:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:33:30.776297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a9edc-f2b1-4a42-acda-98a61fbf5a5f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:30.778570 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-802a9edc-f2b1-4a42-acda-98a61fbf5a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:30.870447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a9edc-f2b1-4a42-acda-98a61fbf5a5f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:30.875425 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb8d64-2fd1-4327-90d6-942df59cc7ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '215c6f6f1aad486f85ce20590d889cbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:30.876254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb8d64-2fd1-4327-90d6-942df59cc7ac service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:30.876564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb8d64-2fd1-4327-90d6-942df59cc7ac service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:30.876949 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb8d64-2fd1-4327-90d6-942df59cc7ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:30.893669 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb8d64-2fd1-4327-90d6-942df59cc7ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:30.895203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb8d64-2fd1-4327-90d6-942df59cc7ac service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:30.909108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1132/2300] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:33:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:33:31.023313 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec896a0-5de8-4566-a63d-5ae86fe095d7 None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:31.023680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec896a0-5de8-4566-a63d-5ae86fe095d7 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:31.023827 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec896a0-5de8-4566-a63d-5ae86fe095d7 None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:31.025206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1169/2301] 10.222.0.42 () {54 vars in 921 bytes} [Tue Jul 19 15:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:33:31.030324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38ebe-a332-4a44-821b-567be761ac45 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:31.030742 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38ebe-a332-4a44-821b-567be761ac45 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:31.031127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38ebe-a332-4a44-821b-567be761ac45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:31.054883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3ed38ebe-a332-4a44-821b-567be761ac45 None None] MFA Rules not processed for user `369bdaded0534142b03ad3cf43a70881`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:33:31.056602 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3ed38ebe-a332-4a44-821b-567be761ac45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:31.127166 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1133/2302] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Jul 19 15:33:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:33:31.142395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec896a0-5de8-4566-a63d-5ae86fe095d7 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:31.144287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5ec896a0-5de8-4566-a63d-5ae86fe095d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:31.160831 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec896a0-5de8-4566-a63d-5ae86fe095d7 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:31.165270 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc5771-91e6-4f48-87de-d67fdef009ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:31.165996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc5771-91e6-4f48-87de-d67fdef009ba service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:31.166344 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc5771-91e6-4f48-87de-d67fdef009ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:31.166972 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc5771-91e6-4f48-87de-d67fdef009ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:31.176834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc5771-91e6-4f48-87de-d67fdef009ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:31.178334 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc5771-91e6-4f48-87de-d67fdef009ba service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:31.188675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1170/2303] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:33:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:33:31.306925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bda48-f33d-44d0-af19-2bccb9cd9e9d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:31.307225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bda48-f33d-44d0-af19-2bccb9cd9e9d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:31.307723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bda48-f33d-44d0-af19-2bccb9cd9e9d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:31.310206 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1134/2304] 10.222.0.42 () {54 vars in 921 bytes} [Tue Jul 19 15:33:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:33:31.318537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49999fe4-c385-4d6e-8f31-847c787dbca3 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:31.319072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49999fe4-c385-4d6e-8f31-847c787dbca3 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:31.319361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49999fe4-c385-4d6e-8f31-847c787dbca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:31.339993 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-49999fe4-c385-4d6e-8f31-847c787dbca3 None None] MFA Rules not processed for user `bff6a2d4bc734904b2615608a86b097d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:33:31.341285 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-49999fe4-c385-4d6e-8f31-847c787dbca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:31.351896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1171/2305] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Jul 19 15:33:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:33:31.363210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bda48-f33d-44d0-af19-2bccb9cd9e9d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:31.375136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bda48-f33d-44d0-af19-2bccb9cd9e9d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:31.380219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ebc0c-e6bf-4840-b171-d8c76786b5eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:31.380873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ebc0c-e6bf-4840-b171-d8c76786b5eb service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:31.381566 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ebc0c-e6bf-4840-b171-d8c76786b5eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:31.381566 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ebc0c-e6bf-4840-b171-d8c76786b5eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:31.394783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ebc0c-e6bf-4840-b171-d8c76786b5eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:31.396858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ebc0c-e6bf-4840-b171-d8c76786b5eb service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:31.411274 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1135/2306] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:33:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:33:34.863076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea213a-912b-4dfa-ba95-1ada9a14cf13 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:34.863703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea213a-912b-4dfa-ba95-1ada9a14cf13 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:34.863703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea213a-912b-4dfa-ba95-1ada9a14cf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:34.905810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-a7ea213a-912b-4dfa-ba95-1ada9a14cf13 None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:33:34.908183 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a7ea213a-912b-4dfa-ba95-1ada9a14cf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:34.917854 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1172/2307] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:33:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:33:34.939079 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ebc0c-e6bf-4840-b171-d8c76786b5eb service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:34.958077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ebc0c-e6bf-4840-b171-d8c76786b5eb service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:34.975353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a332e-f5f3-4f71-8ace-59bda8964b85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:34.976558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a332e-f5f3-4f71-8ace-59bda8964b85 service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:34.977080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a332e-f5f3-4f71-8ace-59bda8964b85 service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:34.977608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a332e-f5f3-4f71-8ace-59bda8964b85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:34.991893 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a332e-f5f3-4f71-8ace-59bda8964b85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:34.994224 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a332e-f5f3-4f71-8ace-59bda8964b85 service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:35.010161 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1136/2308] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:33:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:33:44.884356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=64 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:33:45.001553 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=64 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:33:45.568203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96211ce0-57e5-44f5-b46f-172586f54b63 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:45.571349 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96211ce0-57e5-44f5-b46f-172586f54b63 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:45.572082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96211ce0-57e5-44f5-b46f-172586f54b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:45.616128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-96211ce0-57e5-44f5-b46f-172586f54b63 None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:33:45.617673 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-96211ce0-57e5-44f5-b46f-172586f54b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:45.629172 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1173/2309] 10.222.0.42 () {58 vars in 967 bytes} [Tue Jul 19 15:33:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:33:45.651784 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a332e-f5f3-4f71-8ace-59bda8964b85 service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:45.677203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a332e-f5f3-4f71-8ace-59bda8964b85 service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:45.685930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbf7c4-a1ad-4b02-b84a-af5b8da68d9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:45.686594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbf7c4-a1ad-4b02-b84a-af5b8da68d9a service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:45.686875 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbf7c4-a1ad-4b02-b84a-af5b8da68d9a service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:45.687153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbf7c4-a1ad-4b02-b84a-af5b8da68d9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:45.708586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbf7c4-a1ad-4b02-b84a-af5b8da68d9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:45.708586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbf7c4-a1ad-4b02-b84a-af5b8da68d9a service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:45.717713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1137/2310] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:33:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:33:48.089691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96211ce0-57e5-44f5-b46f-172586f54b63 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:48.091146 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-96211ce0-57e5-44f5-b46f-172586f54b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:48.163582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96211ce0-57e5-44f5-b46f-172586f54b63 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:48.168249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59f266-27e0-4fc8-b220-7ecf7c9a86d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:48.169195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59f266-27e0-4fc8-b220-7ecf7c9a86d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:48.169488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59f266-27e0-4fc8-b220-7ecf7c9a86d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:48.169767 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59f266-27e0-4fc8-b220-7ecf7c9a86d6 admin admin] PATH_INFO: `/v3/users/51afc55d8978440bb5c9e951f9c51c59` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:48.170996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59f266-27e0-4fc8-b220-7ecf7c9a86d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51afc55d8978440bb5c9e951f9c51c59)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:48.174159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59f266-27e0-4fc8-b220-7ecf7c9a86d6 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:48.233535 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6c59f266-27e0-4fc8-b220-7ecf7c9a86d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51afc55d8978440bb5c9e951f9c51c59'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:48.249274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6c59f266-27e0-4fc8-b220-7ecf7c9a86d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51afc55d8978440bb5c9e951f9c51c59'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:48.249993 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6c59f266-27e0-4fc8-b220-7ecf7c9a86d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51afc55d8978440bb5c9e951f9c51c59'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:48.259976 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6c59f266-27e0-4fc8-b220-7ecf7c9a86d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51afc55d8978440bb5c9e951f9c51c59'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:48.260566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6c59f266-27e0-4fc8-b220-7ecf7c9a86d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51afc55d8978440bb5c9e951f9c51c59'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:48.273834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1174/2311] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:33:48 2022] DELETE /identity/v3/users/51afc55d8978440bb5c9e951f9c51c59 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:33:48.490514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbf7c4-a1ad-4b02-b84a-af5b8da68d9a service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:48.509875 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbf7c4-a1ad-4b02-b84a-af5b8da68d9a service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:48.609916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8dddd-ae33-4910-b390-5889877c3c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:48.609916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8dddd-ae33-4910-b390-5889877c3c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:48.610713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8dddd-ae33-4910-b390-5889877c3c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:48.611038 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8dddd-ae33-4910-b390-5889877c3c26 admin admin] PATH_INFO: `/v3/projects/d8ba7f503b3c4afeb9736b241ac2bc98` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:48.612423 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8dddd-ae33-4910-b390-5889877c3c26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8ba7f503b3c4afeb9736b241ac2bc98)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:48.614385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8dddd-ae33-4910-b390-5889877c3c26 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:48.675708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cdb8dddd-ae33-4910-b390-5889877c3c26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8ba7f503b3c4afeb9736b241ac2bc98', 'request_id': 'req-cdb8dddd-ae33-4910-b390-5889877c3c26'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:48.683637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cdb8dddd-ae33-4910-b390-5889877c3c26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8ba7f503b3c4afeb9736b241ac2bc98', 'request_id': 'req-cdb8dddd-ae33-4910-b390-5889877c3c26'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:48.690890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cdb8dddd-ae33-4910-b390-5889877c3c26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8ba7f503b3c4afeb9736b241ac2bc98', 'request_id': 'req-cdb8dddd-ae33-4910-b390-5889877c3c26'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:48.702134 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cdb8dddd-ae33-4910-b390-5889877c3c26 admin admin] The token cache is being invalidate because project d8ba7f503b3c4afeb9736b241ac2bc98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:33:48.702432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cdb8dddd-ae33-4910-b390-5889877c3c26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:48.706546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cdb8dddd-ae33-4910-b390-5889877c3c26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:48.707877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1138/2312] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:33:48 2022] DELETE /identity/v3/projects/d8ba7f503b3c4afeb9736b241ac2bc98 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:33:48.712748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbaada-9a69-4020-b7c0-f69e1b58f7d2 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:48.713039 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbaada-9a69-4020-b7c0-f69e1b58f7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:48.713348 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbaada-9a69-4020-b7c0-f69e1b58f7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:48.736667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-a5dbaada-9a69-4020-b7c0-f69e1b58f7d2 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:33:48.738194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a5dbaada-9a69-4020-b7c0-f69e1b58f7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:48.809126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1175/2313] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:33:48.823489 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bdb51-4f2a-4ba9-b561-1de5f4e3fd35 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:48.823847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bdb51-4f2a-4ba9-b561-1de5f4e3fd35 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:48.824207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bdb51-4f2a-4ba9-b561-1de5f4e3fd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:48.845972 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b98bdb51-4f2a-4ba9-b561-1de5f4e3fd35 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:33:48.847308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b98bdb51-4f2a-4ba9-b561-1de5f4e3fd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:48.856608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1139/2314] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:33:48.860907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbaada-9a69-4020-b7c0-f69e1b58f7d2 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:48.870938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbaada-9a69-4020-b7c0-f69e1b58f7d2 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:48.875523 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfdd4ed-bc49-4650-85c6-7b104ede3561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:48.876323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfdd4ed-bc49-4650-85c6-7b104ede3561 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:48.876603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfdd4ed-bc49-4650-85c6-7b104ede3561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:48.876879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfdd4ed-bc49-4650-85c6-7b104ede3561 admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:48.877650 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfdd4ed-bc49-4650-85c6-7b104ede3561 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:33:48.877957 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfdd4ed-bc49-4650-85c6-7b104ede3561 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:48.879810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfdd4ed-bc49-4650-85c6-7b104ede3561 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:48.889096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:33:48.894204 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1176/2315] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:33:48.898731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bdb51-4f2a-4ba9-b561-1de5f4e3fd35 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:48.909378 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bdb51-4f2a-4ba9-b561-1de5f4e3fd35 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:48.914992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2160a-f2f6-43fb-842e-984f235c07e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:48.915654 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2160a-f2f6-43fb-842e-984f235c07e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:48.915938 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2160a-f2f6-43fb-842e-984f235c07e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:48.916216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2160a-f2f6-43fb-842e-984f235c07e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:48.916912 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2160a-f2f6-43fb-842e-984f235c07e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:48.918863 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2160a-f2f6-43fb-842e-984f235c07e9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:48.946211 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1140/2316] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:33:48 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:33:48.950367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfdd4ed-bc49-4650-85c6-7b104ede3561 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:48.961384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfdd4ed-bc49-4650-85c6-7b104ede3561 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:49.027064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1a34f-f5ec-431d-895c-1dc50da7f1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:49.027740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1a34f-f5ec-431d-895c-1dc50da7f1b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:49.028027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1a34f-f5ec-431d-895c-1dc50da7f1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:49.028304 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1a34f-f5ec-431d-895c-1dc50da7f1b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:49.028991 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1a34f-f5ec-431d-895c-1dc50da7f1b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:49.030976 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1a34f-f5ec-431d-895c-1dc50da7f1b0 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:49.061493 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1177/2317] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:33:48 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:33:49.072953 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2160a-f2f6-43fb-842e-984f235c07e9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:49.091081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2160a-f2f6-43fb-842e-984f235c07e9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:49.099124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fec3c6-6671-4f73-ae49-1b4ccc350322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:49.100085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fec3c6-6671-4f73-ae49-1b4ccc350322 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:49.100521 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fec3c6-6671-4f73-ae49-1b4ccc350322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:49.100945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fec3c6-6671-4f73-ae49-1b4ccc350322 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:49.103983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fec3c6-6671-4f73-ae49-1b4ccc350322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:49.105994 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fec3c6-6671-4f73-ae49-1b4ccc350322 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:49.123513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1141/2318] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:33:49.128090 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1a34f-f5ec-431d-895c-1dc50da7f1b0 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:49.138549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1a34f-f5ec-431d-895c-1dc50da7f1b0 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:49.144174 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf7ff8-fc7b-4fc6-ba65-17fb200d29c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:49.144998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf7ff8-fc7b-4fc6-ba65-17fb200d29c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:49.145363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf7ff8-fc7b-4fc6-ba65-17fb200d29c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:49.145627 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf7ff8-fc7b-4fc6-ba65-17fb200d29c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:49.146579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf7ff8-fc7b-4fc6-ba65-17fb200d29c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:49.149989 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf7ff8-fc7b-4fc6-ba65-17fb200d29c9 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:49.161107 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1178/2319] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:33:49.165572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fec3c6-6671-4f73-ae49-1b4ccc350322 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:49.176175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fec3c6-6671-4f73-ae49-1b4ccc350322 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:49.181275 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9b442-d588-4fed-99f5-6d2046e8a36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:49.182191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9b442-d588-4fed-99f5-6d2046e8a36c admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:49.182477 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9b442-d588-4fed-99f5-6d2046e8a36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:49.182758 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9b442-d588-4fed-99f5-6d2046e8a36c admin admin] PATH_INFO: `/v3/projects/d628a97aa30143ffa30cbb2c26d42c9a/users/389f1fd0230945c18b99e3528e08cd41/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:49.198260 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9b442-d588-4fed-99f5-6d2046e8a36c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d628a97aa30143ffa30cbb2c26d42c9a, user_id=389f1fd0230945c18b99e3528e08cd41, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:49.200975 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9b442-d588-4fed-99f5-6d2046e8a36c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:49.214797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1142/2320] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:33:49 2022] PUT /identity/v3/projects/d628a97aa30143ffa30cbb2c26d42c9a/users/389f1fd0230945c18b99e3528e08cd41/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:33:49.225245 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f8b87-c320-4c88-ba68-a0b985fcbfa2 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:49.225542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f8b87-c320-4c88-ba68-a0b985fcbfa2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:49.225828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f8b87-c320-4c88-ba68-a0b985fcbfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:49.257925 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-9f0f8b87-c320-4c88-ba68-a0b985fcbfa2 None None] MFA Rules not processed for user `389f1fd0230945c18b99e3528e08cd41`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:33:49.259399 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9f0f8b87-c320-4c88-ba68-a0b985fcbfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:49.347041 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1179/2321] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:33:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:33:49.358793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9b442-d588-4fed-99f5-6d2046e8a36c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:49.360856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-21c9b442-d588-4fed-99f5-6d2046e8a36c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:49.395248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:33:49.445551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9b442-d588-4fed-99f5-6d2046e8a36c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:49.449310 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f85669-0ebb-4e6b-96ef-41a563b8d31b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:49.449979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f85669-0ebb-4e6b-96ef-41a563b8d31b service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:49.450265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f85669-0ebb-4e6b-96ef-41a563b8d31b service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:49.450538 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f85669-0ebb-4e6b-96ef-41a563b8d31b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:49.461301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f85669-0ebb-4e6b-96ef-41a563b8d31b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:49.462967 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f85669-0ebb-4e6b-96ef-41a563b8d31b service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:49.477031 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1143/2322] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:33:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:33:49.504175 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f8b87-c320-4c88-ba68-a0b985fcbfa2 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:49.595485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f8b87-c320-4c88-ba68-a0b985fcbfa2 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:49.600706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d35d0-ba69-4ae4-801b-6bec3c441c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:49.601634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d35d0-ba69-4ae4-801b-6bec3c441c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:49.601918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d35d0-ba69-4ae4-801b-6bec3c441c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:49.602196 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d35d0-ba69-4ae4-801b-6bec3c441c6c admin admin] PATH_INFO: `/v3/users/389f1fd0230945c18b99e3528e08cd41` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:49.603430 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d35d0-ba69-4ae4-801b-6bec3c441c6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=389f1fd0230945c18b99e3528e08cd41)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:49.606377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d35d0-ba69-4ae4-801b-6bec3c441c6c admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:49.619556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:33:49.657486 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-116d35d0-ba69-4ae4-801b-6bec3c441c6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '389f1fd0230945c18b99e3528e08cd41'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:49.669358 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-116d35d0-ba69-4ae4-801b-6bec3c441c6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '389f1fd0230945c18b99e3528e08cd41'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:49.670028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-116d35d0-ba69-4ae4-801b-6bec3c441c6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '389f1fd0230945c18b99e3528e08cd41'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:49.677991 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-116d35d0-ba69-4ae4-801b-6bec3c441c6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '389f1fd0230945c18b99e3528e08cd41'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:49.678609 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-116d35d0-ba69-4ae4-801b-6bec3c441c6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '389f1fd0230945c18b99e3528e08cd41'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:49.691680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1180/2323] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:33:49 2022] DELETE /identity/v3/users/389f1fd0230945c18b99e3528e08cd41 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:33:49.696381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5be60b-0856-4b45-b50a-cb0d53973e3f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:49.696904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5be60b-0856-4b45-b50a-cb0d53973e3f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:49.697009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5be60b-0856-4b45-b50a-cb0d53973e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:49.714729 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8d5be60b-0856-4b45-b50a-cb0d53973e3f None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:33:49.715941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8d5be60b-0856-4b45-b50a-cb0d53973e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:49.781826 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1144/2324] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:33:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:33:49.791557 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d35d0-ba69-4ae4-801b-6bec3c441c6c admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:49.792985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-116d35d0-ba69-4ae4-801b-6bec3c441c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:49.859558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d35d0-ba69-4ae4-801b-6bec3c441c6c admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:49.864444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5534606-e211-42c9-b659-172cc0b98f38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:49.865295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5534606-e211-42c9-b659-172cc0b98f38 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:49.865699 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5534606-e211-42c9-b659-172cc0b98f38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:49.866072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5534606-e211-42c9-b659-172cc0b98f38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:49.877428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5534606-e211-42c9-b659-172cc0b98f38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:49.879068 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5534606-e211-42c9-b659-172cc0b98f38 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:49.889910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1181/2325] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:33:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:33:50.180523 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5be60b-0856-4b45-b50a-cb0d53973e3f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:33:50.181969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8d5be60b-0856-4b45-b50a-cb0d53973e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:50.194413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5be60b-0856-4b45-b50a-cb0d53973e3f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:33:50.199250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368bda2d-5fc2-422f-8f76-2f99b2ab9042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:33:50.200133 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bda2d-5fc2-422f-8f76-2f99b2ab9042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:50.200414 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bda2d-5fc2-422f-8f76-2f99b2ab9042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:50.200702 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bda2d-5fc2-422f-8f76-2f99b2ab9042 admin admin] PATH_INFO: `/v3/projects/d628a97aa30143ffa30cbb2c26d42c9a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:50.201906 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368bda2d-5fc2-422f-8f76-2f99b2ab9042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d628a97aa30143ffa30cbb2c26d42c9a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:33:50.203828 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368bda2d-5fc2-422f-8f76-2f99b2ab9042 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:33:50.276960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-368bda2d-5fc2-422f-8f76-2f99b2ab9042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd628a97aa30143ffa30cbb2c26d42c9a', 'request_id': 'req-368bda2d-5fc2-422f-8f76-2f99b2ab9042'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:50.289689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-368bda2d-5fc2-422f-8f76-2f99b2ab9042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd628a97aa30143ffa30cbb2c26d42c9a', 'request_id': 'req-368bda2d-5fc2-422f-8f76-2f99b2ab9042'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:50.296102 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-368bda2d-5fc2-422f-8f76-2f99b2ab9042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd628a97aa30143ffa30cbb2c26d42c9a', 'request_id': 'req-368bda2d-5fc2-422f-8f76-2f99b2ab9042'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:50.307410 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-368bda2d-5fc2-422f-8f76-2f99b2ab9042 admin admin] The token cache is being invalidate because project d628a97aa30143ffa30cbb2c26d42c9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:33:50.307707 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-368bda2d-5fc2-422f-8f76-2f99b2ab9042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:50.308361 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-368bda2d-5fc2-422f-8f76-2f99b2ab9042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:33:50.309937 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1145/2326] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:33:50 2022] DELETE /identity/v3/projects/d628a97aa30143ffa30cbb2c26d42c9a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:33:59.885086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:DELETE=10,keystone:SELECT=48,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:33:59.908025 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fbf4c-68e3-47ec-88c8-f691b0542b09 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:33:59.908390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fbf4c-68e3-47ec-88c8-f691b0542b09 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:33:59.908587 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fbf4c-68e3-47ec-88c8-f691b0542b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:33:59.926312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-cb8fbf4c-68e3-47ec-88c8-f691b0542b09 None None] MFA Rules not processed for user `491d052f3d2947cea318b0753fffd1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:33:59.927534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cb8fbf4c-68e3-47ec-88c8-f691b0542b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:33:59.988329 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1182/2327] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:33:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:34:00.029158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5ba46-041b-45ab-9cc0-14e8f2773748 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:34:00.029631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5ba46-041b-45ab-9cc0-14e8f2773748 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:34:00.030177 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5ba46-041b-45ab-9cc0-14e8f2773748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:34:00.053250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a3f5ba46-041b-45ab-9cc0-14e8f2773748 None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:34:00.055491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a3f5ba46-041b-45ab-9cc0-14e8f2773748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:34:00.101134 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:34:00.128442 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1146/2328] 10.222.0.42 () {58 vars in 967 bytes} [Tue Jul 19 15:34:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:34:00.142563 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fbf4c-68e3-47ec-88c8-f691b0542b09 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:34:00.144121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cb8fbf4c-68e3-47ec-88c8-f691b0542b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:34:00.252582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fbf4c-68e3-47ec-88c8-f691b0542b09 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:34:00.265307 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574bc9d-e2cf-4674-bcd1-910db64e5b39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:34:00.266774 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574bc9d-e2cf-4674-bcd1-910db64e5b39 service swift] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:34:00.267176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574bc9d-e2cf-4674-bcd1-910db64e5b39 service swift] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:34:00.267455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574bc9d-e2cf-4674-bcd1-910db64e5b39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:34:00.277224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5574bc9d-e2cf-4674-bcd1-910db64e5b39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:34:00.278861 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5574bc9d-e2cf-4674-bcd1-910db64e5b39 service swift] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:34:00.288987 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1183/2329] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:34:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:34:03.676418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5ba46-041b-45ab-9cc0-14e8f2773748 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:34:03.677855 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a3f5ba46-041b-45ab-9cc0-14e8f2773748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:34:03.736532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5ba46-041b-45ab-9cc0-14e8f2773748 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:34:03.740115 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85130eb1-a568-4717-bae3-b93b76770192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:34:03.740764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85130eb1-a568-4717-bae3-b93b76770192 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:34:03.741045 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85130eb1-a568-4717-bae3-b93b76770192 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:34:03.741351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85130eb1-a568-4717-bae3-b93b76770192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:34:03.743131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-85130eb1-a568-4717-bae3-b93b76770192 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:34:03.805205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85130eb1-a568-4717-bae3-b93b76770192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:34:03.806574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85130eb1-a568-4717-bae3-b93b76770192 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:34:03.816701 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1147/2330] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:34:03 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:34:10.285351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=57 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:34:13.812466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=57 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:34:18.357673 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e68c69-a33c-4504-b64a-9e6d2bc713f2 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:34:18.358149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e68c69-a33c-4504-b64a-9e6d2bc713f2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:34:18.358253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e68c69-a33c-4504-b64a-9e6d2bc713f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:34:18.376392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-21e68c69-a33c-4504-b64a-9e6d2bc713f2 None None] MFA Rules not processed for user `5507329a6bb84cfbbdac99b85f346ab8`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:34:18.377610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-21e68c69-a33c-4504-b64a-9e6d2bc713f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:34:18.432297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1184/2331] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:34:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:34:28.425684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=25 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:34:50.841955 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6704535-749f-4311-9dbe-43ef635d42e8 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:34:50.842528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6704535-749f-4311-9dbe-43ef635d42e8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:34:50.842528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6704535-749f-4311-9dbe-43ef635d42e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:34:50.863475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a6704535-749f-4311-9dbe-43ef635d42e8 None None] MFA Rules not processed for user `51eb5efc37ca4942831bc4d046f54da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:34:50.864697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a6704535-749f-4311-9dbe-43ef635d42e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:34:50.925374 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1148/2332] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:34:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:34:53.672682 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7f1eb-fe6d-484f-b5d7-12f32f1d4702 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:34:53.673566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7f1eb-fe6d-484f-b5d7-12f32f1d4702 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:34:53.673566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7f1eb-fe6d-484f-b5d7-12f32f1d4702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:34:53.693455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-80d7f1eb-fe6d-484f-b5d7-12f32f1d4702 None None] MFA Rules not processed for user `ddb5e1dec1904ae183e131ba95ed1ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:34:53.694684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-80d7f1eb-fe6d-484f-b5d7-12f32f1d4702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:34:53.759424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1185/2333] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:34:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:35:00.917696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=25 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:35:03.752029 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=25 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:35:14.091750 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6704535-749f-4311-9dbe-43ef635d42e8 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:35:14.101500 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6704535-749f-4311-9dbe-43ef635d42e8 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:35:14.105032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90164f2f-dfc9-4937-a7d4-c0de92263a54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:35:14.105704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90164f2f-dfc9-4937-a7d4-c0de92263a54 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:35:14.105990 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90164f2f-dfc9-4937-a7d4-c0de92263a54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:35:14.107261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90164f2f-dfc9-4937-a7d4-c0de92263a54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:35:14.108878 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-90164f2f-dfc9-4937-a7d4-c0de92263a54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:35:14.180066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90164f2f-dfc9-4937-a7d4-c0de92263a54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:35:14.181559 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90164f2f-dfc9-4937-a7d4-c0de92263a54 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:35:14.192995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1149/2334] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:35:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:35:18.212062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7f1eb-fe6d-484f-b5d7-12f32f1d4702 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:35:18.213521 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-80d7f1eb-fe6d-484f-b5d7-12f32f1d4702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:35:18.273382 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7f1eb-fe6d-484f-b5d7-12f32f1d4702 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:35:18.277099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a6e45-1ccc-4b2a-9fa0-9f9871dc3d81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:35:18.277775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a6e45-1ccc-4b2a-9fa0-9f9871dc3d81 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:35:18.278653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a6e45-1ccc-4b2a-9fa0-9f9871dc3d81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:35:18.278933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a6e45-1ccc-4b2a-9fa0-9f9871dc3d81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:35:18.280728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b41a6e45-1ccc-4b2a-9fa0-9f9871dc3d81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:35:18.294423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a6e45-1ccc-4b2a-9fa0-9f9871dc3d81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:35:18.295898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a6e45-1ccc-4b2a-9fa0-9f9871dc3d81 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:35:18.306865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1186/2335] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:35:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:35:24.188278 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=34 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:35:28.301961 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=26 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:35:33.170900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37ef32-690a-4ad7-8fb1-b33aa70d52e6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:35:33.171407 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37ef32-690a-4ad7-8fb1-b33aa70d52e6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:35:33.171505 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37ef32-690a-4ad7-8fb1-b33aa70d52e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:35:33.188486 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2f37ef32-690a-4ad7-8fb1-b33aa70d52e6 None None] MFA Rules not processed for user `491d052f3d2947cea318b0753fffd1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:35:33.189735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2f37ef32-690a-4ad7-8fb1-b33aa70d52e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:35:33.196709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1150/2336] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:35:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:35:33.208223 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a6e45-1ccc-4b2a-9fa0-9f9871dc3d81 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:35:33.222037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a6e45-1ccc-4b2a-9fa0-9f9871dc3d81 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:35:33.225658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0d9f7-855b-4e58-8955-7dd3b279cb6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:35:33.226302 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0d9f7-855b-4e58-8955-7dd3b279cb6d service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:35:33.226652 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0d9f7-855b-4e58-8955-7dd3b279cb6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:35:33.226951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0d9f7-855b-4e58-8955-7dd3b279cb6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:35:33.228723 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-fab0d9f7-855b-4e58-8955-7dd3b279cb6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:35:33.244080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0d9f7-855b-4e58-8955-7dd3b279cb6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:35:33.245852 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0d9f7-855b-4e58-8955-7dd3b279cb6d service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:35:33.258418 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1187/2337] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:35:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:35:34.751201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f698e5-0a23-4945-8e44-566178b43025 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:35:34.751692 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f698e5-0a23-4945-8e44-566178b43025 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:35:34.751793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f698e5-0a23-4945-8e44-566178b43025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:35:34.770342 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b7f698e5-0a23-4945-8e44-566178b43025 None None] MFA Rules not processed for user `5507329a6bb84cfbbdac99b85f346ab8`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:35:34.771542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b7f698e5-0a23-4945-8e44-566178b43025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:35:34.779143 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1151/2338] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:35:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:35:35.146716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0d9f7-855b-4e58-8955-7dd3b279cb6d service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:35:35.155988 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0d9f7-855b-4e58-8955-7dd3b279cb6d service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:35:35.159522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88e008-d721-44d5-a200-ece5d5ad0bc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:35:35.160170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88e008-d721-44d5-a200-ece5d5ad0bc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:35:35.160454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88e008-d721-44d5-a200-ece5d5ad0bc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:35:35.160731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88e008-d721-44d5-a200-ece5d5ad0bc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:35:35.162556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3f88e008-d721-44d5-a200-ece5d5ad0bc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:35:35.175160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88e008-d721-44d5-a200-ece5d5ad0bc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:35:35.176573 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88e008-d721-44d5-a200-ece5d5ad0bc9 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:35:35.184413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1188/2339] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:35:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:35:35.507245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f698e5-0a23-4945-8e44-566178b43025 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:35:35.509017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b7f698e5-0a23-4945-8e44-566178b43025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:35:35.590571 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f698e5-0a23-4945-8e44-566178b43025 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:35:35.596202 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d725c1-8ed5-4d0c-9ee2-d3002d0a86e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:35:35.596858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d725c1-8ed5-4d0c-9ee2-d3002d0a86e3 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:35:35.597229 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d725c1-8ed5-4d0c-9ee2-d3002d0a86e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:35:35.597462 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d725c1-8ed5-4d0c-9ee2-d3002d0a86e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:35:35.599226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-64d725c1-8ed5-4d0c-9ee2-d3002d0a86e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:35:35.611085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d725c1-8ed5-4d0c-9ee2-d3002d0a86e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:35:35.612686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d725c1-8ed5-4d0c-9ee2-d3002d0a86e3 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:35:35.624615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1152/2340] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:35:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:35:40.581427 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88e008-d721-44d5-a200-ece5d5ad0bc9 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:35:40.582933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3f88e008-d721-44d5-a200-ece5d5ad0bc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:35:40.598454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88e008-d721-44d5-a200-ece5d5ad0bc9 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:35:40.603954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b81a14-24ca-4a15-9a5e-6986e4b26740 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:35:40.604601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b81a14-24ca-4a15-9a5e-6986e4b26740 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:35:40.604885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b81a14-24ca-4a15-9a5e-6986e4b26740 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:35:40.605248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b81a14-24ca-4a15-9a5e-6986e4b26740 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:35:40.607444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e7b81a14-24ca-4a15-9a5e-6986e4b26740 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:35:40.617946 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b81a14-24ca-4a15-9a5e-6986e4b26740 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:35:40.619538 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b81a14-24ca-4a15-9a5e-6986e4b26740 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:35:40.635547 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1189/2341] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:35:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:35:44.323832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d725c1-8ed5-4d0c-9ee2-d3002d0a86e3 service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:35:44.336802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d725c1-8ed5-4d0c-9ee2-d3002d0a86e3 service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:35:44.343017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2209f57-d3a4-492a-b6eb-b73d5edcb73e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:35:44.343650 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2209f57-d3a4-492a-b6eb-b73d5edcb73e service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:35:44.343930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2209f57-d3a4-492a-b6eb-b73d5edcb73e service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:35:44.344205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2209f57-d3a4-492a-b6eb-b73d5edcb73e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:35:44.345955 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d2209f57-d3a4-492a-b6eb-b73d5edcb73e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:35:44.356102 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2209f57-d3a4-492a-b6eb-b73d5edcb73e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:35:44.357732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2209f57-d3a4-492a-b6eb-b73d5edcb73e service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:35:44.366625 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1153/2342] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:35:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:35:44.944207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b81a14-24ca-4a15-9a5e-6986e4b26740 service nova] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:35:44.955281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b81a14-24ca-4a15-9a5e-6986e4b26740 service nova] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:35:44.959654 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60b993-b25a-4382-a8eb-32f0672f8b90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:35:44.960305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60b993-b25a-4382-a8eb-32f0672f8b90 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:35:44.960596 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60b993-b25a-4382-a8eb-32f0672f8b90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:35:44.960867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60b993-b25a-4382-a8eb-32f0672f8b90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:35:44.962711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ed60b993-b25a-4382-a8eb-32f0672f8b90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:35:44.972895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60b993-b25a-4382-a8eb-32f0672f8b90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:35:44.974553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60b993-b25a-4382-a8eb-32f0672f8b90 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:35:44.983358 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1190/2343] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:35:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:35:54.362506 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=44 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:35:54.979109 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=24 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:06.154785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2209f57-d3a4-492a-b6eb-b73d5edcb73e service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:06.166716 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2209f57-d3a4-492a-b6eb-b73d5edcb73e service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:06.173125 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4defc-f18b-4974-8648-fe7efec16bab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:06.173813 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4defc-f18b-4974-8648-fe7efec16bab service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:06.174118 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4defc-f18b-4974-8648-fe7efec16bab service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:06.174486 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4defc-f18b-4974-8648-fe7efec16bab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:06.176615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-67f4defc-f18b-4974-8648-fe7efec16bab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:06.190561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4defc-f18b-4974-8648-fe7efec16bab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:06.192163 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4defc-f18b-4974-8648-fe7efec16bab service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:06.208526 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1154/2344] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:36:11.061730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60b993-b25a-4382-a8eb-32f0672f8b90 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:11.073246 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60b993-b25a-4382-a8eb-32f0672f8b90 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:11.077859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34564c9-64cf-46e7-871a-e275be5d9f08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:11.078514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34564c9-64cf-46e7-871a-e275be5d9f08 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:11.078845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34564c9-64cf-46e7-871a-e275be5d9f08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:11.079118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34564c9-64cf-46e7-871a-e275be5d9f08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:11.080898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a34564c9-64cf-46e7-871a-e275be5d9f08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:11.148705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34564c9-64cf-46e7-871a-e275be5d9f08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:11.150624 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34564c9-64cf-46e7-871a-e275be5d9f08 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:11.161228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1191/2345] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:12.173624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95852b68-a5e7-4ca2-8421-cbab4cdf6bff None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:12.174116 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95852b68-a5e7-4ca2-8421-cbab4cdf6bff None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:12.174219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95852b68-a5e7-4ca2-8421-cbab4cdf6bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:12.192963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-95852b68-a5e7-4ca2-8421-cbab4cdf6bff None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:12.194209 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-95852b68-a5e7-4ca2-8421-cbab4cdf6bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:12.201941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1155/2346] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:36:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:12.260306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34564c9-64cf-46e7-871a-e275be5d9f08 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:12.273258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34564c9-64cf-46e7-871a-e275be5d9f08 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:12.282430 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5044e5-1842-470f-942b-4065d68d156f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:12.283123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5044e5-1842-470f-942b-4065d68d156f service swift] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:12.283410 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5044e5-1842-470f-942b-4065d68d156f service swift] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:12.283691 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5044e5-1842-470f-942b-4065d68d156f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:12.293384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5044e5-1842-470f-942b-4065d68d156f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:12.295108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5044e5-1842-470f-942b-4065d68d156f service swift] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:12.304457 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1192/2347] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:36:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:36:13.007297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95852b68-a5e7-4ca2-8421-cbab4cdf6bff None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:13.033184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95852b68-a5e7-4ca2-8421-cbab4cdf6bff None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:13.045266 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4ec53-769c-46c5-949b-ac4686f1a6c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:13.046033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4ec53-769c-46c5-949b-ac4686f1a6c2 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:13.046597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4ec53-769c-46c5-949b-ac4686f1a6c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:13.047005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4ec53-769c-46c5-949b-ac4686f1a6c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:13.053574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-01c4ec53-769c-46c5-949b-ac4686f1a6c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:13.097040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4ec53-769c-46c5-949b-ac4686f1a6c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:13.099634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4ec53-769c-46c5-949b-ac4686f1a6c2 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:13.115717 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1156/2348] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:15.266107 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5044e5-1842-470f-942b-4065d68d156f service swift] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:15.267891 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cf5044e5-1842-470f-942b-4065d68d156f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:15.292692 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5044e5-1842-470f-942b-4065d68d156f service swift] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:15.304459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8f9ef-0a1d-4cf2-8a57-0b682223f8cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:15.305319 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8f9ef-0a1d-4cf2-8a57-0b682223f8cb service placement] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:15.305860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8f9ef-0a1d-4cf2-8a57-0b682223f8cb service placement] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:15.306462 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8f9ef-0a1d-4cf2-8a57-0b682223f8cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:15.309616 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1ec8f9ef-0a1d-4cf2-8a57-0b682223f8cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:15.330251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4ec53-769c-46c5-949b-ac4686f1a6c2 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:15.331760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-01c4ec53-769c-46c5-949b-ac4686f1a6c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:15.341396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8f9ef-0a1d-4cf2-8a57-0b682223f8cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:15.343037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8f9ef-0a1d-4cf2-8a57-0b682223f8cb service placement] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:15.358208 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4ec53-769c-46c5-949b-ac4686f1a6c2 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:15.366046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1193/2349] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:15.367847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c30c6-6fb6-45bb-a3a2-54b6b40692aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:15.368973 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c30c6-6fb6-45bb-a3a2-54b6b40692aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:15.369451 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c30c6-6fb6-45bb-a3a2-54b6b40692aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:15.369871 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c30c6-6fb6-45bb-a3a2-54b6b40692aa admin admin] PATH_INFO: `/v3/users/51eb5efc37ca4942831bc4d046f54da6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:15.373400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c30c6-6fb6-45bb-a3a2-54b6b40692aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=51eb5efc37ca4942831bc4d046f54da6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:15.375895 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c30c6-6fb6-45bb-a3a2-54b6b40692aa admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:15.459667 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-210c30c6-6fb6-45bb-a3a2-54b6b40692aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51eb5efc37ca4942831bc4d046f54da6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:15.472984 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-210c30c6-6fb6-45bb-a3a2-54b6b40692aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51eb5efc37ca4942831bc4d046f54da6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:15.473777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-210c30c6-6fb6-45bb-a3a2-54b6b40692aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51eb5efc37ca4942831bc4d046f54da6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:15.474381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-210c30c6-6fb6-45bb-a3a2-54b6b40692aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51eb5efc37ca4942831bc4d046f54da6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:15.486987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-210c30c6-6fb6-45bb-a3a2-54b6b40692aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51eb5efc37ca4942831bc4d046f54da6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:15.502709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1157/2350] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:36:15 2022] DELETE /identity/v3/users/51eb5efc37ca4942831bc4d046f54da6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:15.724484 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8f9ef-0a1d-4cf2-8a57-0b682223f8cb service placement] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:15.743669 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8f9ef-0a1d-4cf2-8a57-0b682223f8cb service placement] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:15.817401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364b95f-fef9-4e33-a05b-63a0d957dc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:15.818846 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364b95f-fef9-4e33-a05b-63a0d957dc93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:15.819066 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364b95f-fef9-4e33-a05b-63a0d957dc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:15.819323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364b95f-fef9-4e33-a05b-63a0d957dc93 admin admin] PATH_INFO: `/v3/projects/7d55d02d809143f682698fbb494f65f4` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:15.820744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364b95f-fef9-4e33-a05b-63a0d957dc93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d55d02d809143f682698fbb494f65f4)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:15.823017 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364b95f-fef9-4e33-a05b-63a0d957dc93 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:15.890340 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6364b95f-fef9-4e33-a05b-63a0d957dc93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d55d02d809143f682698fbb494f65f4', 'request_id': 'req-6364b95f-fef9-4e33-a05b-63a0d957dc93'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:15.898726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6364b95f-fef9-4e33-a05b-63a0d957dc93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d55d02d809143f682698fbb494f65f4', 'request_id': 'req-6364b95f-fef9-4e33-a05b-63a0d957dc93'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:15.907724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6364b95f-fef9-4e33-a05b-63a0d957dc93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d55d02d809143f682698fbb494f65f4', 'request_id': 'req-6364b95f-fef9-4e33-a05b-63a0d957dc93'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:15.918007 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:15.924793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6364b95f-fef9-4e33-a05b-63a0d957dc93 admin admin] The token cache is being invalidate because project 7d55d02d809143f682698fbb494f65f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:36:15.925625 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6364b95f-fef9-4e33-a05b-63a0d957dc93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:15.927258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6364b95f-fef9-4e33-a05b-63a0d957dc93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:15.929123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1194/2351] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:36:15 2022] DELETE /identity/v3/projects/7d55d02d809143f682698fbb494f65f4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:15.934880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c52029-2445-48dd-8c17-4b5f0a10ccd1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:15.935625 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c52029-2445-48dd-8c17-4b5f0a10ccd1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:15.936125 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c52029-2445-48dd-8c17-4b5f0a10ccd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:15.966116 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-96c52029-2445-48dd-8c17-4b5f0a10ccd1 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:15.968162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-96c52029-2445-48dd-8c17-4b5f0a10ccd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:16.059564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1158/2352] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:16.061309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613616f1-8610-410e-98c6-407036992c35 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:16.061622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613616f1-8610-410e-98c6-407036992c35 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:16.062170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613616f1-8610-410e-98c6-407036992c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:16.092552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-613616f1-8610-410e-98c6-407036992c35 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:16.095708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-613616f1-8610-410e-98c6-407036992c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:16.106699 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1195/2353] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:16.110900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c52029-2445-48dd-8c17-4b5f0a10ccd1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:16.123634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c52029-2445-48dd-8c17-4b5f0a10ccd1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:16.129279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c19dcb-0d66-4755-9095-224c6fd9af59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:16.130365 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c19dcb-0d66-4755-9095-224c6fd9af59 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:16.130877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c19dcb-0d66-4755-9095-224c6fd9af59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:16.131365 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c19dcb-0d66-4755-9095-224c6fd9af59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:16.135998 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c19dcb-0d66-4755-9095-224c6fd9af59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:36:16.137264 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c19dcb-0d66-4755-9095-224c6fd9af59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:16.141394 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c19dcb-0d66-4755-9095-224c6fd9af59 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:16.159910 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1159/2354] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:36:16.164333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613616f1-8610-410e-98c6-407036992c35 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:16.175127 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613616f1-8610-410e-98c6-407036992c35 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:16.180787 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da58c9-e6ab-44ee-a50e-a36a13611f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:16.181531 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da58c9-e6ab-44ee-a50e-a36a13611f0a admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:16.181818 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da58c9-e6ab-44ee-a50e-a36a13611f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:16.182106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da58c9-e6ab-44ee-a50e-a36a13611f0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:16.183081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da58c9-e6ab-44ee-a50e-a36a13611f0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:16.184995 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da58c9-e6ab-44ee-a50e-a36a13611f0a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:16.220917 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1196/2355] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:36:16 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:36:16.221764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c19dcb-0d66-4755-9095-224c6fd9af59 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:16.236389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c19dcb-0d66-4755-9095-224c6fd9af59 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:16.279576 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:16.317353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a34ac-140b-4345-869a-bfb58bbe4820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:16.318043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a34ac-140b-4345-869a-bfb58bbe4820 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:16.318328 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a34ac-140b-4345-869a-bfb58bbe4820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:16.318608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a34ac-140b-4345-869a-bfb58bbe4820 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:16.319320 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a34ac-140b-4345-869a-bfb58bbe4820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:16.321276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a34ac-140b-4345-869a-bfb58bbe4820 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:16.344091 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1160/2356] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:36:16 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:36:16.348269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da58c9-e6ab-44ee-a50e-a36a13611f0a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:16.360460 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da58c9-e6ab-44ee-a50e-a36a13611f0a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:16.365393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cea6fb-d75d-48ce-a064-6d2c28b4dd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:16.366378 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cea6fb-d75d-48ce-a064-6d2c28b4dd91 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:16.366818 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cea6fb-d75d-48ce-a064-6d2c28b4dd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:16.367253 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cea6fb-d75d-48ce-a064-6d2c28b4dd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:16.368222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cea6fb-d75d-48ce-a064-6d2c28b4dd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:16.370027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cea6fb-d75d-48ce-a064-6d2c28b4dd91 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:16.382589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1197/2357] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:36:16.391870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a34ac-140b-4345-869a-bfb58bbe4820 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:16.404073 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a34ac-140b-4345-869a-bfb58bbe4820 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:16.410836 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1341b-a204-4a56-b265-ebc1a37703ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:16.411521 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1341b-a204-4a56-b265-ebc1a37703ab admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:16.411806 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1341b-a204-4a56-b265-ebc1a37703ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:16.412088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1341b-a204-4a56-b265-ebc1a37703ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:16.412830 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1341b-a204-4a56-b265-ebc1a37703ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:16.414488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1341b-a204-4a56-b265-ebc1a37703ab admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:16.431980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1161/2358] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:16.436375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cea6fb-d75d-48ce-a064-6d2c28b4dd91 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:16.454742 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cea6fb-d75d-48ce-a064-6d2c28b4dd91 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:16.461685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55883991-6f42-43e2-a7c6-958341653855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:16.462900 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55883991-6f42-43e2-a7c6-958341653855 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:16.463373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55883991-6f42-43e2-a7c6-958341653855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:16.463897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55883991-6f42-43e2-a7c6-958341653855 admin admin] PATH_INFO: `/v3/projects/921a4687972d45a5a5e91116265b9926/users/9d8d48c0731d4aa5ad38a35e196053d2/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:16.486353 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55883991-6f42-43e2-a7c6-958341653855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=921a4687972d45a5a5e91116265b9926, user_id=9d8d48c0731d4aa5ad38a35e196053d2, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:16.489753 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55883991-6f42-43e2-a7c6-958341653855 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:16.517986 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1198/2359] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:16 2022] PUT /identity/v3/projects/921a4687972d45a5a5e91116265b9926/users/9d8d48c0731d4aa5ad38a35e196053d2/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:16.531367 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35404d58-c0c5-431c-b038-45e9f3a73516 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:16.531888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35404d58-c0c5-431c-b038-45e9f3a73516 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:16.532285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35404d58-c0c5-431c-b038-45e9f3a73516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:16.562305 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-35404d58-c0c5-431c-b038-45e9f3a73516 None None] MFA Rules not processed for user `9d8d48c0731d4aa5ad38a35e196053d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:16.563542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-35404d58-c0c5-431c-b038-45e9f3a73516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:16.641410 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1162/2360] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:16.653772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55883991-6f42-43e2-a7c6-958341653855 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:16.655928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-55883991-6f42-43e2-a7c6-958341653855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:16.756268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55883991-6f42-43e2-a7c6-958341653855 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:16.765321 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddcf7f-a049-4958-9aa6-0f38eaeb81b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:16.766202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddcf7f-a049-4958-9aa6-0f38eaeb81b9 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:16.766548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddcf7f-a049-4958-9aa6-0f38eaeb81b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:16.767323 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddcf7f-a049-4958-9aa6-0f38eaeb81b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:16.778260 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddcf7f-a049-4958-9aa6-0f38eaeb81b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:16.780115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddcf7f-a049-4958-9aa6-0f38eaeb81b9 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:16.805531 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1199/2361] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:24.474952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35404d58-c0c5-431c-b038-45e9f3a73516 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:24.540845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35404d58-c0c5-431c-b038-45e9f3a73516 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:24.549914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e9e94-25ff-4d26-9418-ef7af15f3389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:24.551029 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e9e94-25ff-4d26-9418-ef7af15f3389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:24.551423 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e9e94-25ff-4d26-9418-ef7af15f3389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:24.551804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e9e94-25ff-4d26-9418-ef7af15f3389 admin admin] PATH_INFO: `/v3/users/9d8d48c0731d4aa5ad38a35e196053d2` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:24.553550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e9e94-25ff-4d26-9418-ef7af15f3389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d8d48c0731d4aa5ad38a35e196053d2)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:24.555486 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e9e94-25ff-4d26-9418-ef7af15f3389 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:24.609664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:24.623151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5d9e9e94-25ff-4d26-9418-ef7af15f3389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d8d48c0731d4aa5ad38a35e196053d2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:24.639377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5d9e9e94-25ff-4d26-9418-ef7af15f3389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d8d48c0731d4aa5ad38a35e196053d2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:24.640192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5d9e9e94-25ff-4d26-9418-ef7af15f3389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d8d48c0731d4aa5ad38a35e196053d2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:24.640800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5d9e9e94-25ff-4d26-9418-ef7af15f3389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d8d48c0731d4aa5ad38a35e196053d2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:24.649937 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5d9e9e94-25ff-4d26-9418-ef7af15f3389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d8d48c0731d4aa5ad38a35e196053d2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:24.669313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1163/2362] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:36:24 2022] DELETE /identity/v3/users/9d8d48c0731d4aa5ad38a35e196053d2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:24.672996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5b899-1b2d-4360-b343-66366ef8a7a8 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:24.673511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5b899-1b2d-4360-b343-66366ef8a7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:24.673609 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5b899-1b2d-4360-b343-66366ef8a7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:24.691830 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-22e5b899-1b2d-4360-b343-66366ef8a7a8 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:24.693086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-22e5b899-1b2d-4360-b343-66366ef8a7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:24.756093 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1200/2363] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:24.766509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e9e94-25ff-4d26-9418-ef7af15f3389 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:24.767869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5d9e9e94-25ff-4d26-9418-ef7af15f3389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:24.833433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e9e94-25ff-4d26-9418-ef7af15f3389 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:24.837735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bd8e0-d9a2-44fe-943c-7917b4179af3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:24.838567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bd8e0-d9a2-44fe-943c-7917b4179af3 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:24.838928 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bd8e0-d9a2-44fe-943c-7917b4179af3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:24.839834 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bd8e0-d9a2-44fe-943c-7917b4179af3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:24.853895 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bd8e0-d9a2-44fe-943c-7917b4179af3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:24.856232 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bd8e0-d9a2-44fe-943c-7917b4179af3 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:24.866791 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1164/2364] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:25.151954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5b899-1b2d-4360-b343-66366ef8a7a8 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:25.153840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-22e5b899-1b2d-4360-b343-66366ef8a7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:25.166830 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5b899-1b2d-4360-b343-66366ef8a7a8 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:25.171822 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becd73f-23f7-4e58-a900-145383be41cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:25.172698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becd73f-23f7-4e58-a900-145383be41cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:25.173130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becd73f-23f7-4e58-a900-145383be41cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:25.173431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becd73f-23f7-4e58-a900-145383be41cc admin admin] PATH_INFO: `/v3/projects/921a4687972d45a5a5e91116265b9926` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:25.174640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9becd73f-23f7-4e58-a900-145383be41cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=921a4687972d45a5a5e91116265b9926)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:25.176573 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9becd73f-23f7-4e58-a900-145383be41cc admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:25.194103 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:25.228525 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9becd73f-23f7-4e58-a900-145383be41cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '921a4687972d45a5a5e91116265b9926', 'request_id': 'req-9becd73f-23f7-4e58-a900-145383be41cc'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:25.234234 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9becd73f-23f7-4e58-a900-145383be41cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '921a4687972d45a5a5e91116265b9926', 'request_id': 'req-9becd73f-23f7-4e58-a900-145383be41cc'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:25.241084 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9becd73f-23f7-4e58-a900-145383be41cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '921a4687972d45a5a5e91116265b9926', 'request_id': 'req-9becd73f-23f7-4e58-a900-145383be41cc'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:25.264195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9becd73f-23f7-4e58-a900-145383be41cc admin admin] The token cache is being invalidate because project 921a4687972d45a5a5e91116265b9926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:36:25.264481 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9becd73f-23f7-4e58-a900-145383be41cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:25.265102 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9becd73f-23f7-4e58-a900-145383be41cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:25.267114 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1201/2365] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:36:25 2022] DELETE /identity/v3/projects/921a4687972d45a5a5e91116265b9926 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:25.272351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f5b1d-1f64-4e68-a0ac-d42004dc8107 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:25.272571 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f5b1d-1f64-4e68-a0ac-d42004dc8107 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:25.272851 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f5b1d-1f64-4e68-a0ac-d42004dc8107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:25.291023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-070f5b1d-1f64-4e68-a0ac-d42004dc8107 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:25.292435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-070f5b1d-1f64-4e68-a0ac-d42004dc8107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:25.375739 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1165/2366] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:25.384148 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20beb365-2969-4d6d-8c12-a40ef4d3340b None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:25.384448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20beb365-2969-4d6d-8c12-a40ef4d3340b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:25.384734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20beb365-2969-4d6d-8c12-a40ef4d3340b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:25.411385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-20beb365-2969-4d6d-8c12-a40ef4d3340b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:25.413775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-20beb365-2969-4d6d-8c12-a40ef4d3340b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:25.423554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1202/2367] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:25.427460 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f5b1d-1f64-4e68-a0ac-d42004dc8107 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:25.439262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f5b1d-1f64-4e68-a0ac-d42004dc8107 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:25.444652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781bec01-ece3-4d6e-ab2c-d432b1b2ea18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:25.445651 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bec01-ece3-4d6e-ab2c-d432b1b2ea18 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:25.446096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bec01-ece3-4d6e-ab2c-d432b1b2ea18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:25.446530 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bec01-ece3-4d6e-ab2c-d432b1b2ea18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:25.447441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781bec01-ece3-4d6e-ab2c-d432b1b2ea18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:36:25.448631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781bec01-ece3-4d6e-ab2c-d432b1b2ea18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:25.450957 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781bec01-ece3-4d6e-ab2c-d432b1b2ea18 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:25.467471 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1166/2368] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:36:25.468613 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20beb365-2969-4d6d-8c12-a40ef4d3340b None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:25.480908 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20beb365-2969-4d6d-8c12-a40ef4d3340b None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:25.487139 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3cd81-8fa5-4ebc-be54-cbb248711545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:25.487818 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3cd81-8fa5-4ebc-be54-cbb248711545 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:25.488093 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3cd81-8fa5-4ebc-be54-cbb248711545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:25.488368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3cd81-8fa5-4ebc-be54-cbb248711545 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:25.489078 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3cd81-8fa5-4ebc-be54-cbb248711545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:25.491033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3cd81-8fa5-4ebc-be54-cbb248711545 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:25.533375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1203/2369] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:36:25 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:36:25.534693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781bec01-ece3-4d6e-ab2c-d432b1b2ea18 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:25.546416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781bec01-ece3-4d6e-ab2c-d432b1b2ea18 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:25.594120 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:25.612731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6216906-f3e2-4103-9354-677a7f4794f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:25.613515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6216906-f3e2-4103-9354-677a7f4794f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:25.613797 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6216906-f3e2-4103-9354-677a7f4794f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:25.614072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6216906-f3e2-4103-9354-677a7f4794f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:25.614778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6216906-f3e2-4103-9354-677a7f4794f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:25.616735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6216906-f3e2-4103-9354-677a7f4794f9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:25.639060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1167/2370] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:36:25 2022] POST /identity/v3/users => generated 283 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:36:25.643263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3cd81-8fa5-4ebc-be54-cbb248711545 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:25.655011 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3cd81-8fa5-4ebc-be54-cbb248711545 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:25.660033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2ee12-0faf-41d9-9ba8-50f35552b3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:25.660710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2ee12-0faf-41d9-9ba8-50f35552b3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:25.660998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2ee12-0faf-41d9-9ba8-50f35552b3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:25.661311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2ee12-0faf-41d9-9ba8-50f35552b3e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:25.662062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2ee12-0faf-41d9-9ba8-50f35552b3e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:25.663677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2ee12-0faf-41d9-9ba8-50f35552b3e1 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:25.678607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1204/2371] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:25.683476 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6216906-f3e2-4103-9354-677a7f4794f9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:25.694888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6216906-f3e2-4103-9354-677a7f4794f9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:25.699912 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e452d-db2b-4f45-8953-6dc6ccd5dbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:25.700584 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e452d-db2b-4f45-8953-6dc6ccd5dbc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:25.700873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e452d-db2b-4f45-8953-6dc6ccd5dbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:25.701177 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e452d-db2b-4f45-8953-6dc6ccd5dbc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:25.701923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e452d-db2b-4f45-8953-6dc6ccd5dbc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:25.703534 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e452d-db2b-4f45-8953-6dc6ccd5dbc1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:25.714476 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1168/2372] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:25.718746 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2ee12-0faf-41d9-9ba8-50f35552b3e1 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:25.729440 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2ee12-0faf-41d9-9ba8-50f35552b3e1 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:25.734705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca720db3-5d36-4c6f-92fb-ffa2c72b30d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:25.735600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca720db3-5d36-4c6f-92fb-ffa2c72b30d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:25.735888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca720db3-5d36-4c6f-92fb-ffa2c72b30d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:25.736168 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca720db3-5d36-4c6f-92fb-ffa2c72b30d3 admin admin] PATH_INFO: `/v3/projects/51f5a58d1aed457c91d0f7f346503a15/users/9788b856b6d94a08b99e7ab751e2a9f1/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:25.751503 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca720db3-5d36-4c6f-92fb-ffa2c72b30d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51f5a58d1aed457c91d0f7f346503a15, user_id=9788b856b6d94a08b99e7ab751e2a9f1, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:25.754214 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca720db3-5d36-4c6f-92fb-ffa2c72b30d3 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:25.768040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1205/2373] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:25 2022] PUT /identity/v3/projects/51f5a58d1aed457c91d0f7f346503a15/users/9788b856b6d94a08b99e7ab751e2a9f1/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:25.780066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd17bd-0bbd-4c8c-96d8-9aec7114b012 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:25.780362 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd17bd-0bbd-4c8c-96d8-9aec7114b012 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:25.780648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd17bd-0bbd-4c8c-96d8-9aec7114b012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:25.807946 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7bdd17bd-0bbd-4c8c-96d8-9aec7114b012 None None] MFA Rules not processed for user `9788b856b6d94a08b99e7ab751e2a9f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:25.809178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7bdd17bd-0bbd-4c8c-96d8-9aec7114b012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:25.875217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1169/2374] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:25.886303 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca720db3-5d36-4c6f-92fb-ffa2c72b30d3 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:25.888335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ca720db3-5d36-4c6f-92fb-ffa2c72b30d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:25.957599 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca720db3-5d36-4c6f-92fb-ffa2c72b30d3 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:25.961484 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41171a-f6aa-42b2-8c4b-5ec56d58a2ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:25.962144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41171a-f6aa-42b2-8c4b-5ec56d58a2ae service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:25.962431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41171a-f6aa-42b2-8c4b-5ec56d58a2ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:25.962721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41171a-f6aa-42b2-8c4b-5ec56d58a2ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:25.972767 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41171a-f6aa-42b2-8c4b-5ec56d58a2ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:25.974638 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41171a-f6aa-42b2-8c4b-5ec56d58a2ae service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:25.986267 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1206/2375] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:25 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:36:27.841704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39650df0-efac-488c-b438-0d01f041f1f4 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:27.842182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39650df0-efac-488c-b438-0d01f041f1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:27.842367 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39650df0-efac-488c-b438-0d01f041f1f4 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:27.843819 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1170/2376] 10.222.0.42 () {54 vars in 921 bytes} [Tue Jul 19 15:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:36:27.852059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832546c-8668-4d6a-8ba8-83c4ef72f1d2 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:27.852492 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832546c-8668-4d6a-8ba8-83c4ef72f1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:27.852670 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832546c-8668-4d6a-8ba8-83c4ef72f1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:27.875477 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-9832546c-8668-4d6a-8ba8-83c4ef72f1d2 None None] MFA Rules not processed for user `bff6a2d4bc734904b2615608a86b097d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:27.876718 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9832546c-8668-4d6a-8ba8-83c4ef72f1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:27.934228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:27.972046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1207/2377] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Jul 19 15:36:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:36:27.984839 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39650df0-efac-488c-b438-0d01f041f1f4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:27.986733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-39650df0-efac-488c-b438-0d01f041f1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:28.052586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39650df0-efac-488c-b438-0d01f041f1f4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:28.056249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4530af5-887e-4408-9579-f51f11680143 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '215c6f6f1aad486f85ce20590d889cbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:28.057072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4530af5-887e-4408-9579-f51f11680143 service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:28.057950 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4530af5-887e-4408-9579-f51f11680143 service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:28.058303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4530af5-887e-4408-9579-f51f11680143 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:28.070070 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4530af5-887e-4408-9579-f51f11680143 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:28.071480 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4530af5-887e-4408-9579-f51f11680143 service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:28.083663 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1171/2378] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:36:28.155088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3872a4-fc73-49da-a4ec-e7662266140e None None] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:28.155380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3872a4-fc73-49da-a4ec-e7662266140e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:28.155664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3872a4-fc73-49da-a4ec-e7662266140e None None] PATH_INFO: `/` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:28.157101 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1208/2379] 10.222.0.42 () {54 vars in 921 bytes} [Tue Jul 19 15:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:36:28.161989 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a749e-00d3-4f61-8992-8e16b42a866b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:28.162279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a749e-00d3-4f61-8992-8e16b42a866b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:28.162558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a749e-00d3-4f61-8992-8e16b42a866b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:28.181535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-623a749e-00d3-4f61-8992-8e16b42a866b None None] MFA Rules not processed for user `369bdaded0534142b03ad3cf43a70881`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:28.183319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-623a749e-00d3-4f61-8992-8e16b42a866b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:28.238243 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1172/2380] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Jul 19 15:36:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:28.251081 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3872a4-fc73-49da-a4ec-e7662266140e None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:28.252907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-6a3872a4-fc73-49da-a4ec-e7662266140e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:28.266814 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3872a4-fc73-49da-a4ec-e7662266140e None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:28.270591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979375f-9c88-41fd-bd13-9056dbeb9130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:28.271257 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979375f-9c88-41fd-bd13-9056dbeb9130 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:28.271535 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979375f-9c88-41fd-bd13-9056dbeb9130 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:28.271806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979375f-9c88-41fd-bd13-9056dbeb9130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:28.278793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979375f-9c88-41fd-bd13-9056dbeb9130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:28.280209 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979375f-9c88-41fd-bd13-9056dbeb9130 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:28.287811 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1209/2381] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:28.371603 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae589f-7399-4411-8b65-1443e57aee86 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:28.371918 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae589f-7399-4411-8b65-1443e57aee86 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:28.372199 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae589f-7399-4411-8b65-1443e57aee86 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:28.373662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1173/2382] 10.222.0.42 () {54 vars in 921 bytes} [Tue Jul 19 15:36:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 15:36:28.379042 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2116ab-5d36-4717-8a86-2b2a5baac678 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:28.379330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2116ab-5d36-4717-8a86-2b2a5baac678 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:28.379631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2116ab-5d36-4717-8a86-2b2a5baac678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:28.397807 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-cd2116ab-5d36-4717-8a86-2b2a5baac678 None None] MFA Rules not processed for user `bff6a2d4bc734904b2615608a86b097d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:28.398992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cd2116ab-5d36-4717-8a86-2b2a5baac678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:28.408847 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1210/2383] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Jul 19 15:36:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:28.423185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae589f-7399-4411-8b65-1443e57aee86 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:28.425635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-57ae589f-7399-4411-8b65-1443e57aee86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:28.436375 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:28.441428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae589f-7399-4411-8b65-1443e57aee86 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:28.445354 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14661978-e779-41de-b391-6418eddc00fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:28.445995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14661978-e779-41de-b391-6418eddc00fc service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:28.446282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14661978-e779-41de-b391-6418eddc00fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:28.446570 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14661978-e779-41de-b391-6418eddc00fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:28.457510 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14661978-e779-41de-b391-6418eddc00fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:28.458950 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14661978-e779-41de-b391-6418eddc00fc service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:28.470695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1174/2384] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:32.355514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dfa4e-f3b6-4e80-8139-d121026646ab None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:32.356103 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dfa4e-f3b6-4e80-8139-d121026646ab None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:32.356228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dfa4e-f3b6-4e80-8139-d121026646ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:32.380699 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ce3dfa4e-f3b6-4e80-8139-d121026646ab None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:32.382076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ce3dfa4e-f3b6-4e80-8139-d121026646ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:32.445228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1211/2385] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:36:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:32.458250 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14661978-e779-41de-b391-6418eddc00fc service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:32.459702 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-14661978-e779-41de-b391-6418eddc00fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:32.533088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14661978-e779-41de-b391-6418eddc00fc service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:32.538236 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc671e8-7e58-45ac-b11a-e5ca47693151 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:32.539010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc671e8-7e58-45ac-b11a-e5ca47693151 service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:32.539300 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc671e8-7e58-45ac-b11a-e5ca47693151 service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:32.539581 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc671e8-7e58-45ac-b11a-e5ca47693151 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:32.550225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc671e8-7e58-45ac-b11a-e5ca47693151 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:32.552130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc671e8-7e58-45ac-b11a-e5ca47693151 service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:32.563621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1175/2386] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:36:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:36:34.658114 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dfa4e-f3b6-4e80-8139-d121026646ab None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:34.737509 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dfa4e-f3b6-4e80-8139-d121026646ab None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:34.742326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793aba8-81d3-454b-99a7-a20a38c3d5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:34.743293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793aba8-81d3-454b-99a7-a20a38c3d5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:34.743576 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793aba8-81d3-454b-99a7-a20a38c3d5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:34.743853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793aba8-81d3-454b-99a7-a20a38c3d5e7 admin admin] PATH_INFO: `/v3/users/9788b856b6d94a08b99e7ab751e2a9f1` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:34.745422 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793aba8-81d3-454b-99a7-a20a38c3d5e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9788b856b6d94a08b99e7ab751e2a9f1)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:34.747363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793aba8-81d3-454b-99a7-a20a38c3d5e7 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:34.802823 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-0793aba8-81d3-454b-99a7-a20a38c3d5e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9788b856b6d94a08b99e7ab751e2a9f1'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:34.818485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:34.821184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-0793aba8-81d3-454b-99a7-a20a38c3d5e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9788b856b6d94a08b99e7ab751e2a9f1'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:34.822101 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-0793aba8-81d3-454b-99a7-a20a38c3d5e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9788b856b6d94a08b99e7ab751e2a9f1'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:34.834179 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-0793aba8-81d3-454b-99a7-a20a38c3d5e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9788b856b6d94a08b99e7ab751e2a9f1'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:34.835369 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-0793aba8-81d3-454b-99a7-a20a38c3d5e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9788b856b6d94a08b99e7ab751e2a9f1'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:34.855304 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1212/2387] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:36:34 2022] DELETE /identity/v3/users/9788b856b6d94a08b99e7ab751e2a9f1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:34.859948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21f53f-a7cf-4b85-b3a7-f86d54fe3fb3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:34.860468 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21f53f-a7cf-4b85-b3a7-f86d54fe3fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:34.860554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21f53f-a7cf-4b85-b3a7-f86d54fe3fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:34.878923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0a21f53f-a7cf-4b85-b3a7-f86d54fe3fb3 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:34.887575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0a21f53f-a7cf-4b85-b3a7-f86d54fe3fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:34.993560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1176/2388] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:35.004762 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793aba8-81d3-454b-99a7-a20a38c3d5e7 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:35.006504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0793aba8-81d3-454b-99a7-a20a38c3d5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:35.079705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793aba8-81d3-454b-99a7-a20a38c3d5e7 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:35.084203 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d818a8-ac20-4f6f-b010-6c7e959f1712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:35.085125 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d818a8-ac20-4f6f-b010-6c7e959f1712 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:35.085556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d818a8-ac20-4f6f-b010-6c7e959f1712 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:35.085985 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d818a8-ac20-4f6f-b010-6c7e959f1712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:35.097324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d818a8-ac20-4f6f-b010-6c7e959f1712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:35.098880 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d818a8-ac20-4f6f-b010-6c7e959f1712 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:35.109214 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1213/2389] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:35.461359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21f53f-a7cf-4b85-b3a7-f86d54fe3fb3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:35.462968 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0a21f53f-a7cf-4b85-b3a7-f86d54fe3fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:35.479196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21f53f-a7cf-4b85-b3a7-f86d54fe3fb3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:35.484300 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44861a-5642-465b-9bee-ad1b64d15402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:35.485341 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44861a-5642-465b-9bee-ad1b64d15402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:35.485694 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44861a-5642-465b-9bee-ad1b64d15402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:35.486040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44861a-5642-465b-9bee-ad1b64d15402 admin admin] PATH_INFO: `/v3/projects/51f5a58d1aed457c91d0f7f346503a15` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:35.487430 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44861a-5642-465b-9bee-ad1b64d15402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51f5a58d1aed457c91d0f7f346503a15)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:35.489980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44861a-5642-465b-9bee-ad1b64d15402 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:35.593886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2f44861a-5642-465b-9bee-ad1b64d15402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51f5a58d1aed457c91d0f7f346503a15', 'request_id': 'req-2f44861a-5642-465b-9bee-ad1b64d15402'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:35.602331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2f44861a-5642-465b-9bee-ad1b64d15402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51f5a58d1aed457c91d0f7f346503a15', 'request_id': 'req-2f44861a-5642-465b-9bee-ad1b64d15402'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:35.610436 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2f44861a-5642-465b-9bee-ad1b64d15402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51f5a58d1aed457c91d0f7f346503a15', 'request_id': 'req-2f44861a-5642-465b-9bee-ad1b64d15402'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:35.625051 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2f44861a-5642-465b-9bee-ad1b64d15402 admin admin] The token cache is being invalidate because project 51f5a58d1aed457c91d0f7f346503a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:36:35.625378 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2f44861a-5642-465b-9bee-ad1b64d15402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:35.626078 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2f44861a-5642-465b-9bee-ad1b64d15402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:35.632394 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1177/2390] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:36:35 2022] DELETE /identity/v3/projects/51f5a58d1aed457c91d0f7f346503a15 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:35.633875 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e20d5-f9f4-40c9-afa2-3808beef661f None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:35.634272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e20d5-f9f4-40c9-afa2-3808beef661f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:35.634618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e20d5-f9f4-40c9-afa2-3808beef661f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:35.692466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-e56e20d5-f9f4-40c9-afa2-3808beef661f None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:35.698392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e56e20d5-f9f4-40c9-afa2-3808beef661f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:35.762671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1214/2391] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:36:35.770741 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ee3bb-3224-4d5a-bb16-80e77f469d07 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:35.771123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ee3bb-3224-4d5a-bb16-80e77f469d07 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:35.771457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ee3bb-3224-4d5a-bb16-80e77f469d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:35.779873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:35.793240 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-959ee3bb-3224-4d5a-bb16-80e77f469d07 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:35.794984 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-959ee3bb-3224-4d5a-bb16-80e77f469d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:35.803821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1178/2392] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:35.808311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e20d5-f9f4-40c9-afa2-3808beef661f None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:35.818396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e20d5-f9f4-40c9-afa2-3808beef661f None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:35.823570 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b238d3-eed7-4d47-a199-f5fdedb1723f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:35.824258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b238d3-eed7-4d47-a199-f5fdedb1723f admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:35.824545 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b238d3-eed7-4d47-a199-f5fdedb1723f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:35.824823 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b238d3-eed7-4d47-a199-f5fdedb1723f admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:35.825598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b238d3-eed7-4d47-a199-f5fdedb1723f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:36:35.825912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b238d3-eed7-4d47-a199-f5fdedb1723f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:35.827544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b238d3-eed7-4d47-a199-f5fdedb1723f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:35.844549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1215/2393] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:36:35.848839 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ee3bb-3224-4d5a-bb16-80e77f469d07 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:35.860899 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ee3bb-3224-4d5a-bb16-80e77f469d07 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:35.865604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ab5ca-df70-4774-b597-a333cec63689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:35.866377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ab5ca-df70-4774-b597-a333cec63689 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:35.866655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ab5ca-df70-4774-b597-a333cec63689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:35.866939 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ab5ca-df70-4774-b597-a333cec63689 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:35.867696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ab5ca-df70-4774-b597-a333cec63689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:35.869859 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ab5ca-df70-4774-b597-a333cec63689 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:35.899784 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1179/2394] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:36:35 2022] POST /identity/v3/projects => generated 355 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:36:35.904283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b238d3-eed7-4d47-a199-f5fdedb1723f admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:35.914610 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b238d3-eed7-4d47-a199-f5fdedb1723f admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:35.978549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70659d3c-fec1-41f7-ae3d-8f34d554cace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:35.979773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70659d3c-fec1-41f7-ae3d-8f34d554cace admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:35.980176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70659d3c-fec1-41f7-ae3d-8f34d554cace admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:35.980505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70659d3c-fec1-41f7-ae3d-8f34d554cace admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:35.981526 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70659d3c-fec1-41f7-ae3d-8f34d554cace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:35.983578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70659d3c-fec1-41f7-ae3d-8f34d554cace admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:36.005332 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:36.033838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1216/2395] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:36:35 2022] POST /identity/v3/users => generated 278 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:36:36.035765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ab5ca-df70-4774-b597-a333cec63689 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:36.051188 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ab5ca-df70-4774-b597-a333cec63689 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:36.056760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d518c-c067-47ec-90dd-a9c694d6e1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:36.058636 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d518c-c067-47ec-90dd-a9c694d6e1fc admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:36.059120 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d518c-c067-47ec-90dd-a9c694d6e1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:36.059556 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d518c-c067-47ec-90dd-a9c694d6e1fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:36.060795 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d518c-c067-47ec-90dd-a9c694d6e1fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:36.062854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d518c-c067-47ec-90dd-a9c694d6e1fc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:36.078407 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1180/2396] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:36.081630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70659d3c-fec1-41f7-ae3d-8f34d554cace admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:36.094298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70659d3c-fec1-41f7-ae3d-8f34d554cace admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:36.099145 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1218fd1-d3cd-4f97-bad4-d3adce67dca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:36.099821 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1218fd1-d3cd-4f97-bad4-d3adce67dca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:36.100106 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1218fd1-d3cd-4f97-bad4-d3adce67dca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:36.100384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1218fd1-d3cd-4f97-bad4-d3adce67dca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:36.101115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1218fd1-d3cd-4f97-bad4-d3adce67dca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:36.102752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1218fd1-d3cd-4f97-bad4-d3adce67dca9 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:36.117274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1217/2397] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:36.121744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d518c-c067-47ec-90dd-a9c694d6e1fc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:36.134302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d518c-c067-47ec-90dd-a9c694d6e1fc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:36.145252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2d065-ec6a-4e1b-a86c-c6437910ddab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:36.146233 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2d065-ec6a-4e1b-a86c-c6437910ddab admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:36.146598 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2d065-ec6a-4e1b-a86c-c6437910ddab admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:36.147574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2d065-ec6a-4e1b-a86c-c6437910ddab admin admin] PATH_INFO: `/v3/projects/3ae6bba82e4942aca4dba4cf1eb25913/users/e264c4ea6b734dccb449d782d3c57353/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:36.164031 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2d065-ec6a-4e1b-a86c-c6437910ddab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ae6bba82e4942aca4dba4cf1eb25913, user_id=e264c4ea6b734dccb449d782d3c57353, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:36.166754 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2d065-ec6a-4e1b-a86c-c6437910ddab admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:36.180078 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1181/2398] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:36 2022] PUT /identity/v3/projects/3ae6bba82e4942aca4dba4cf1eb25913/users/e264c4ea6b734dccb449d782d3c57353/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:36.192119 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bddf84-2e19-4ab7-9883-d39bbd4bc4e4 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:36.192426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bddf84-2e19-4ab7-9883-d39bbd4bc4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:36.192707 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bddf84-2e19-4ab7-9883-d39bbd4bc4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:36.230813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-85bddf84-2e19-4ab7-9883-d39bbd4bc4e4 None None] MFA Rules not processed for user `e264c4ea6b734dccb449d782d3c57353`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:36.232129 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-85bddf84-2e19-4ab7-9883-d39bbd4bc4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:36.317928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1218/2399] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:36 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:36.321747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2d065-ec6a-4e1b-a86c-c6437910ddab admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:36.392463 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2d065-ec6a-4e1b-a86c-c6437910ddab admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:36.396796 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde67be8-3dc5-4384-bd0d-19b5fc625da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:36.399372 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde67be8-3dc5-4384-bd0d-19b5fc625da7 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:36.399683 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde67be8-3dc5-4384-bd0d-19b5fc625da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:36.399989 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde67be8-3dc5-4384-bd0d-19b5fc625da7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:36.400976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde67be8-3dc5-4384-bd0d-19b5fc625da7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:36.403213 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde67be8-3dc5-4384-bd0d-19b5fc625da7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:36.451805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1182/2400] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:36:36 2022] POST /identity/v3/projects => generated 357 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:36:36.456805 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bddf84-2e19-4ab7-9883-d39bbd4bc4e4 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:36.472062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bddf84-2e19-4ab7-9883-d39bbd4bc4e4 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:36.538920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa9e32-4547-4198-8cfb-15bfcac00790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:36.539621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa9e32-4547-4198-8cfb-15bfcac00790 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:36.539907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa9e32-4547-4198-8cfb-15bfcac00790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:36.540189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa9e32-4547-4198-8cfb-15bfcac00790 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:36.540941 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa9e32-4547-4198-8cfb-15bfcac00790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:36.542973 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa9e32-4547-4198-8cfb-15bfcac00790 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:36.573760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1219/2401] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:36:36 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:36:36.574826 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde67be8-3dc5-4384-bd0d-19b5fc625da7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:36.585357 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde67be8-3dc5-4384-bd0d-19b5fc625da7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:36.590087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cad726-b395-44b3-9b04-b59be7d9a72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:36.590751 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cad726-b395-44b3-9b04-b59be7d9a72c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:36.591041 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cad726-b395-44b3-9b04-b59be7d9a72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:36.591324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cad726-b395-44b3-9b04-b59be7d9a72c admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:36.592095 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cad726-b395-44b3-9b04-b59be7d9a72c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:36.593742 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cad726-b395-44b3-9b04-b59be7d9a72c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:36.604823 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1183/2402] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:36:36.609287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa9e32-4547-4198-8cfb-15bfcac00790 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:36.622793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa9e32-4547-4198-8cfb-15bfcac00790 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:36.628709 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9d088-6eab-4b12-a96e-675ffc5225ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:36.629726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9d088-6eab-4b12-a96e-675ffc5225ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:36.630019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9d088-6eab-4b12-a96e-675ffc5225ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:36.630305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9d088-6eab-4b12-a96e-675ffc5225ed admin admin] PATH_INFO: `/v3/projects/a26222767ad1431b89d65bcb96658a0b/users/bc15061650b94e8a969eb6c80112d04a/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:36.647668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9d088-6eab-4b12-a96e-675ffc5225ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26222767ad1431b89d65bcb96658a0b, user_id=bc15061650b94e8a969eb6c80112d04a, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:36.650460 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9d088-6eab-4b12-a96e-675ffc5225ed admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:36.663194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1220/2403] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:36 2022] PUT /identity/v3/projects/a26222767ad1431b89d65bcb96658a0b/users/bc15061650b94e8a969eb6c80112d04a/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:36.667484 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cad726-b395-44b3-9b04-b59be7d9a72c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:36.678291 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cad726-b395-44b3-9b04-b59be7d9a72c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:36.752358 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9ca8c-1a3c-4a8a-b9a7-6eaf58ae9f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:36.753408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9ca8c-1a3c-4a8a-b9a7-6eaf58ae9f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:36.753764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9ca8c-1a3c-4a8a-b9a7-6eaf58ae9f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:36.754267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9ca8c-1a3c-4a8a-b9a7-6eaf58ae9f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:36.755055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9ca8c-1a3c-4a8a-b9a7-6eaf58ae9f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:36.756858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9ca8c-1a3c-4a8a-b9a7-6eaf58ae9f35 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:36.769167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1184/2404] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:36.772937 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9d088-6eab-4b12-a96e-675ffc5225ed admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:36.784083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9d088-6eab-4b12-a96e-675ffc5225ed admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:36.789938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1424b7-9d4e-4610-8120-13130882772f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:36.791542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1424b7-9d4e-4610-8120-13130882772f admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:36.791825 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1424b7-9d4e-4610-8120-13130882772f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:36.792100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1424b7-9d4e-4610-8120-13130882772f admin admin] PATH_INFO: `/v3/projects/a26222767ad1431b89d65bcb96658a0b/users/bc15061650b94e8a969eb6c80112d04a/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:36.794162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1424b7-9d4e-4610-8120-13130882772f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26222767ad1431b89d65bcb96658a0b, user_id=bc15061650b94e8a969eb6c80112d04a, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:36.796831 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1424b7-9d4e-4610-8120-13130882772f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:36.810090 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1221/2405] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:36 2022] PUT /identity/v3/projects/a26222767ad1431b89d65bcb96658a0b/users/bc15061650b94e8a969eb6c80112d04a/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:36.821283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10489c8a-77c0-4875-90c4-b91df2c30568 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:36.821588 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10489c8a-77c0-4875-90c4-b91df2c30568 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:36.821878 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10489c8a-77c0-4875-90c4-b91df2c30568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:36.842472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:36.852540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-10489c8a-77c0-4875-90c4-b91df2c30568 None None] MFA Rules not processed for user `bc15061650b94e8a969eb6c80112d04a`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:36.856026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-10489c8a-77c0-4875-90c4-b91df2c30568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:36.956078 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1185/2406] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:36 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:36.968070 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1424b7-9d4e-4610-8120-13130882772f admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:36.970079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ca1424b7-9d4e-4610-8120-13130882772f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:37.007130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:37.042000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1424b7-9d4e-4610-8120-13130882772f admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:37.046308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726dc63-fa8b-4c5d-b5e3-9290dadf5602 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:37.047047 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726dc63-fa8b-4c5d-b5e3-9290dadf5602 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:37.047333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726dc63-fa8b-4c5d-b5e3-9290dadf5602 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:37.047739 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726dc63-fa8b-4c5d-b5e3-9290dadf5602 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:37.059238 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726dc63-fa8b-4c5d-b5e3-9290dadf5602 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:37.060706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726dc63-fa8b-4c5d-b5e3-9290dadf5602 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:37.072189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1222/2407] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:36 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:37.154722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10489c8a-77c0-4875-90c4-b91df2c30568 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:37.164373 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10489c8a-77c0-4875-90c4-b91df2c30568 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:37.186952 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8418f6-f52a-408b-99c4-34ce5a95fec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:37.187647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8418f6-f52a-408b-99c4-34ce5a95fec7 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:37.187932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8418f6-f52a-408b-99c4-34ce5a95fec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:37.188209 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8418f6-f52a-408b-99c4-34ce5a95fec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:37.296843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8418f6-f52a-408b-99c4-34ce5a95fec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:37.298345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8418f6-f52a-408b-99c4-34ce5a95fec7 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:37.316638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1186/2408] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:37 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:47.066592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=13 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:47.303749 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=57 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:48.106314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726dc63-fa8b-4c5d-b5e3-9290dadf5602 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:48.107811 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f726dc63-fa8b-4c5d-b5e3-9290dadf5602 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:48.164873 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726dc63-fa8b-4c5d-b5e3-9290dadf5602 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:48.169123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a0e63-1a0f-418b-bfd2-8940544660ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:48.169803 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a0e63-1a0f-418b-bfd2-8940544660ef service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:48.170087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a0e63-1a0f-418b-bfd2-8940544660ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:48.170363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a0e63-1a0f-418b-bfd2-8940544660ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:48.172151 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9f3a0e63-1a0f-418b-bfd2-8940544660ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:48.248103 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3a0e63-1a0f-418b-bfd2-8940544660ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:48.250130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3a0e63-1a0f-418b-bfd2-8940544660ef service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:48.264197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1223/2409] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:49.269242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e215ad-eca1-4a7d-aad6-4a683c5f538e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:49.269747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e215ad-eca1-4a7d-aad6-4a683c5f538e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:49.270137 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e215ad-eca1-4a7d-aad6-4a683c5f538e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:49.304896 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-34e215ad-eca1-4a7d-aad6-4a683c5f538e None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:49.306353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-34e215ad-eca1-4a7d-aad6-4a683c5f538e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:49.372543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1187/2410] 10.222.0.42 () {58 vars in 967 bytes} [Tue Jul 19 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:49.387918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a0e63-1a0f-418b-bfd2-8940544660ef service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:49.389705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9f3a0e63-1a0f-418b-bfd2-8940544660ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:49.476136 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a0e63-1a0f-418b-bfd2-8940544660ef service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:49.481356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dc8de-46d6-47f2-bb11-575581841215 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:49.482009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dc8de-46d6-47f2-bb11-575581841215 service swift] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:49.482293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dc8de-46d6-47f2-bb11-575581841215 service swift] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:49.482570 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dc8de-46d6-47f2-bb11-575581841215 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:49.491115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51dc8de-46d6-47f2-bb11-575581841215 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:49.492712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51dc8de-46d6-47f2-bb11-575581841215 service swift] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:49.502129 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1224/2411] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:36:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:36:52.412441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e215ad-eca1-4a7d-aad6-4a683c5f538e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:52.414773 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-34e215ad-eca1-4a7d-aad6-4a683c5f538e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:52.436953 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e215ad-eca1-4a7d-aad6-4a683c5f538e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:52.442618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a0a5c-7296-4c19-a6b3-836757bd8ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:52.446493 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a0a5c-7296-4c19-a6b3-836757bd8ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:52.446764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a0a5c-7296-4c19-a6b3-836757bd8ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:52.447113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a0a5c-7296-4c19-a6b3-836757bd8ee1 admin admin] PATH_INFO: `/v3/users/491d052f3d2947cea318b0753fffd1fa` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:52.448608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a0a5c-7296-4c19-a6b3-836757bd8ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=491d052f3d2947cea318b0753fffd1fa)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:52.451112 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a0a5c-7296-4c19-a6b3-836757bd8ee1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:52.516854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ab6a0a5c-7296-4c19-a6b3-836757bd8ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '491d052f3d2947cea318b0753fffd1fa'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:52.538927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ab6a0a5c-7296-4c19-a6b3-836757bd8ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '491d052f3d2947cea318b0753fffd1fa'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:52.539736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ab6a0a5c-7296-4c19-a6b3-836757bd8ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '491d052f3d2947cea318b0753fffd1fa'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:52.540611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ab6a0a5c-7296-4c19-a6b3-836757bd8ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '491d052f3d2947cea318b0753fffd1fa'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:52.553770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ab6a0a5c-7296-4c19-a6b3-836757bd8ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '491d052f3d2947cea318b0753fffd1fa'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:52.577313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1188/2412] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:36:52 2022] DELETE /identity/v3/users/491d052f3d2947cea318b0753fffd1fa => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:52.832180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dc8de-46d6-47f2-bb11-575581841215 service swift] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:52.853662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dc8de-46d6-47f2-bb11-575581841215 service swift] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:52.868333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a0a5c-7296-4c19-a6b3-836757bd8ee1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:52.870041 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ab6a0a5c-7296-4c19-a6b3-836757bd8ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:52.914220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:52.935807 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29b6ff-d821-4545-b51f-551ec62515cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:52.937283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29b6ff-d821-4545-b51f-551ec62515cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:52.937593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29b6ff-d821-4545-b51f-551ec62515cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:52.938086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29b6ff-d821-4545-b51f-551ec62515cf admin admin] PATH_INFO: `/v3/projects/8c3bc3582aa7420a9a1fcccacff362bd` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:52.939508 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29b6ff-d821-4545-b51f-551ec62515cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c3bc3582aa7420a9a1fcccacff362bd)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:52.941656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29b6ff-d821-4545-b51f-551ec62515cf admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:52.955018 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a0a5c-7296-4c19-a6b3-836757bd8ee1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:52.959911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190551ea-42f1-4b66-8c5d-2eeff51d0b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:52.960793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190551ea-42f1-4b66-8c5d-2eeff51d0b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:52.961082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190551ea-42f1-4b66-8c5d-2eeff51d0b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:52.961397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190551ea-42f1-4b66-8c5d-2eeff51d0b66 admin admin] PATH_INFO: `/v3/users/e264c4ea6b734dccb449d782d3c57353` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:52.962924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190551ea-42f1-4b66-8c5d-2eeff51d0b66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e264c4ea6b734dccb449d782d3c57353)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:52.965069 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190551ea-42f1-4b66-8c5d-2eeff51d0b66 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:52.991846 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:53.007806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1c29b6ff-d821-4545-b51f-551ec62515cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c3bc3582aa7420a9a1fcccacff362bd', 'request_id': 'req-1c29b6ff-d821-4545-b51f-551ec62515cf'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:53.017504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1c29b6ff-d821-4545-b51f-551ec62515cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c3bc3582aa7420a9a1fcccacff362bd', 'request_id': 'req-1c29b6ff-d821-4545-b51f-551ec62515cf'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:53.024569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1c29b6ff-d821-4545-b51f-551ec62515cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c3bc3582aa7420a9a1fcccacff362bd', 'request_id': 'req-1c29b6ff-d821-4545-b51f-551ec62515cf'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:53.030397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-190551ea-42f1-4b66-8c5d-2eeff51d0b66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e264c4ea6b734dccb449d782d3c57353'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:53.042599 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1c29b6ff-d821-4545-b51f-551ec62515cf admin admin] The token cache is being invalidate because project 8c3bc3582aa7420a9a1fcccacff362bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:36:53.042928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1c29b6ff-d821-4545-b51f-551ec62515cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:53.044037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1c29b6ff-d821-4545-b51f-551ec62515cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:53.046023 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1225/2413] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:36:52 2022] DELETE /identity/v3/projects/8c3bc3582aa7420a9a1fcccacff362bd => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:53.051554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-190551ea-42f1-4b66-8c5d-2eeff51d0b66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e264c4ea6b734dccb449d782d3c57353'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:53.052297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-190551ea-42f1-4b66-8c5d-2eeff51d0b66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e264c4ea6b734dccb449d782d3c57353'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:53.052754 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b49a1d-ea36-49fe-a596-0d47bc523181 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:53.052986 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-190551ea-42f1-4b66-8c5d-2eeff51d0b66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e264c4ea6b734dccb449d782d3c57353'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:53.053213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b49a1d-ea36-49fe-a596-0d47bc523181 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:53.053921 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b49a1d-ea36-49fe-a596-0d47bc523181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:53.064514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-190551ea-42f1-4b66-8c5d-2eeff51d0b66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e264c4ea6b734dccb449d782d3c57353'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:53.086388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-33b49a1d-ea36-49fe-a596-0d47bc523181 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:53.086388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-33b49a1d-ea36-49fe-a596-0d47bc523181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:53.090018 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1189/2414] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:36:52 2022] DELETE /identity/v3/users/e264c4ea6b734dccb449d782d3c57353 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:53.094357 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871338d-48ef-4914-a8ba-ad6f80b632a1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:53.094718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871338d-48ef-4914-a8ba-ad6f80b632a1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:53.094999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871338d-48ef-4914-a8ba-ad6f80b632a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:53.115647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b871338d-48ef-4914-a8ba-ad6f80b632a1 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:53.116966 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b871338d-48ef-4914-a8ba-ad6f80b632a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:53.174461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1226/2415] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:53.185037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20351559-aafd-47b9-ad3b-6001b2662762 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:53.185579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20351559-aafd-47b9-ad3b-6001b2662762 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:53.186074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20351559-aafd-47b9-ad3b-6001b2662762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:53.198167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1190/2416] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:36:53.210641 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871338d-48ef-4914-a8ba-ad6f80b632a1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:53.214155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b871338d-48ef-4914-a8ba-ad6f80b632a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:53.216168 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-20351559-aafd-47b9-ad3b-6001b2662762 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:53.218401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-20351559-aafd-47b9-ad3b-6001b2662762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:53.232409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1227/2417] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:36:53.233486 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20351559-aafd-47b9-ad3b-6001b2662762 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:53.253079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20351559-aafd-47b9-ad3b-6001b2662762 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:53.267499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96346201-0139-40e4-bdb1-da42043b485d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:53.268898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96346201-0139-40e4-bdb1-da42043b485d admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:53.271796 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96346201-0139-40e4-bdb1-da42043b485d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:53.272317 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96346201-0139-40e4-bdb1-da42043b485d admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:53.276335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96346201-0139-40e4-bdb1-da42043b485d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:36:53.276704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96346201-0139-40e4-bdb1-da42043b485d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:53.281949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96346201-0139-40e4-bdb1-da42043b485d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:53.299513 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1228/2418] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:36:53.304049 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96346201-0139-40e4-bdb1-da42043b485d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:53.316647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96346201-0139-40e4-bdb1-da42043b485d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:53.324701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9636da6-2b97-4162-aa3f-7f1015c437eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:53.324926 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871338d-48ef-4914-a8ba-ad6f80b632a1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:53.325043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9636da6-2b97-4162-aa3f-7f1015c437eb admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:53.326711 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9636da6-2b97-4162-aa3f-7f1015c437eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:53.326964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9636da6-2b97-4162-aa3f-7f1015c437eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:53.327722 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9636da6-2b97-4162-aa3f-7f1015c437eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:53.329701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9636da6-2b97-4162-aa3f-7f1015c437eb admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:53.331426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045d1e2-ed82-498b-8487-f526c6202d8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:53.331575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045d1e2-ed82-498b-8487-f526c6202d8c service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:53.331772 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045d1e2-ed82-498b-8487-f526c6202d8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:53.332050 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045d1e2-ed82-498b-8487-f526c6202d8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:53.344351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d045d1e2-ed82-498b-8487-f526c6202d8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:53.345836 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d045d1e2-ed82-498b-8487-f526c6202d8c service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:53.358225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1191/2419] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:36:53.362822 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1229/2420] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:36:53 2022] POST /identity/v3/projects => generated 405 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:36:53.366749 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045d1e2-ed82-498b-8487-f526c6202d8c service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:53.377368 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045d1e2-ed82-498b-8487-f526c6202d8c service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:53.416142 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:53.440113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fde26c-7041-4ea3-9a7e-e37fd2f85c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:53.440736 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fde26c-7041-4ea3-9a7e-e37fd2f85c53 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:53.441100 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fde26c-7041-4ea3-9a7e-e37fd2f85c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:53.441692 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fde26c-7041-4ea3-9a7e-e37fd2f85c53 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:53.443110 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fde26c-7041-4ea3-9a7e-e37fd2f85c53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:53.445816 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fde26c-7041-4ea3-9a7e-e37fd2f85c53 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:53.471510 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1192/2421] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:36:53 2022] POST /identity/v3/users => generated 303 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:36:53.472716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9636da6-2b97-4162-aa3f-7f1015c437eb admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:53.483990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9636da6-2b97-4162-aa3f-7f1015c437eb admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:53.488799 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d657594-d4ea-49e8-9b72-320eafb9b0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:53.489499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d657594-d4ea-49e8-9b72-320eafb9b0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:53.489807 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d657594-d4ea-49e8-9b72-320eafb9b0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:53.490123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d657594-d4ea-49e8-9b72-320eafb9b0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:53.490938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d657594-d4ea-49e8-9b72-320eafb9b0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:53.492850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d657594-d4ea-49e8-9b72-320eafb9b0c1 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:53.504594 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1230/2422] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:53.508422 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fde26c-7041-4ea3-9a7e-e37fd2f85c53 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:53.519222 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fde26c-7041-4ea3-9a7e-e37fd2f85c53 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:53.524015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a2306-8d41-40a9-ac3b-1f39ccb38f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:53.524684 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a2306-8d41-40a9-ac3b-1f39ccb38f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:53.524980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a2306-8d41-40a9-ac3b-1f39ccb38f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:53.525294 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a2306-8d41-40a9-ac3b-1f39ccb38f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:53.526035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a2306-8d41-40a9-ac3b-1f39ccb38f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:53.528334 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a2306-8d41-40a9-ac3b-1f39ccb38f75 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:53.539215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1193/2423] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:53.544243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d657594-d4ea-49e8-9b72-320eafb9b0c1 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:53.565545 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d657594-d4ea-49e8-9b72-320eafb9b0c1 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:53.570377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fde52f-0331-4000-979d-c6172218d43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:53.571291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fde52f-0331-4000-979d-c6172218d43e admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:53.571580 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fde52f-0331-4000-979d-c6172218d43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:53.571859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fde52f-0331-4000-979d-c6172218d43e admin admin] PATH_INFO: `/v3/projects/6046006ba4f3451694b4ebe24a076a27/users/5a23b34c425f4764b342026a38b1ea07/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:53.587668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fde52f-0331-4000-979d-c6172218d43e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6046006ba4f3451694b4ebe24a076a27, user_id=5a23b34c425f4764b342026a38b1ea07, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:53.591091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fde52f-0331-4000-979d-c6172218d43e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:53.593449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:53.607710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1231/2424] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:53 2022] PUT /identity/v3/projects/6046006ba4f3451694b4ebe24a076a27/users/5a23b34c425f4764b342026a38b1ea07/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:53.617743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d44af-ac1e-4c1f-81d1-25092428d848 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:53.618153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d44af-ac1e-4c1f-81d1-25092428d848 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:53.618436 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d44af-ac1e-4c1f-81d1-25092428d848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:53.645930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-258d44af-ac1e-4c1f-81d1-25092428d848 None None] MFA Rules not processed for user `5a23b34c425f4764b342026a38b1ea07`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:53.647139 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-258d44af-ac1e-4c1f-81d1-25092428d848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:53.684933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fde52f-0331-4000-979d-c6172218d43e admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:53.687051 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e6fde52f-0331-4000-979d-c6172218d43e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:53.735035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1194/2425] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:53 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:53.735996 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d44af-ac1e-4c1f-81d1-25092428d848 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:53.755649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fde52f-0331-4000-979d-c6172218d43e admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:53.760807 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3829cdc-34ed-4f7c-b2b1-1b552c209562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:53.761940 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3829cdc-34ed-4f7c-b2b1-1b552c209562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:53.762626 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3829cdc-34ed-4f7c-b2b1-1b552c209562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:53.764028 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3829cdc-34ed-4f7c-b2b1-1b552c209562 admin admin] PATH_INFO: `/v3/projects/3ae6bba82e4942aca4dba4cf1eb25913` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:53.766534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3829cdc-34ed-4f7c-b2b1-1b552c209562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ae6bba82e4942aca4dba4cf1eb25913)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:53.769627 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3829cdc-34ed-4f7c-b2b1-1b552c209562 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:53.810870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d44af-ac1e-4c1f-81d1-25092428d848 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:53.816057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db97ac7-a2c3-4d94-a8e5-d50821b965cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:53.816873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db97ac7-a2c3-4d94-a8e5-d50821b965cb admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:53.817812 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db97ac7-a2c3-4d94-a8e5-d50821b965cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:53.818201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db97ac7-a2c3-4d94-a8e5-d50821b965cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:53.818966 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db97ac7-a2c3-4d94-a8e5-d50821b965cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:53.820956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db97ac7-a2c3-4d94-a8e5-d50821b965cb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:53.832754 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c3829cdc-34ed-4f7c-b2b1-1b552c209562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ae6bba82e4942aca4dba4cf1eb25913', 'request_id': 'req-c3829cdc-34ed-4f7c-b2b1-1b552c209562'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:53.841851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c3829cdc-34ed-4f7c-b2b1-1b552c209562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ae6bba82e4942aca4dba4cf1eb25913', 'request_id': 'req-c3829cdc-34ed-4f7c-b2b1-1b552c209562'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:53.851633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c3829cdc-34ed-4f7c-b2b1-1b552c209562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ae6bba82e4942aca4dba4cf1eb25913', 'request_id': 'req-c3829cdc-34ed-4f7c-b2b1-1b552c209562'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:53.858222 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1195/2426] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:36:53 2022] POST /identity/v3/projects => generated 405 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:36:53.859405 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db97ac7-a2c3-4d94-a8e5-d50821b965cb admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:53.873602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c3829cdc-34ed-4f7c-b2b1-1b552c209562 admin admin] The token cache is being invalidate because project 3ae6bba82e4942aca4dba4cf1eb25913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:36:53.874185 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c3829cdc-34ed-4f7c-b2b1-1b552c209562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:53.874997 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c3829cdc-34ed-4f7c-b2b1-1b552c209562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:53.880005 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1232/2427] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:36:53 2022] DELETE /identity/v3/projects/3ae6bba82e4942aca4dba4cf1eb25913 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:53.881374 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3829cdc-34ed-4f7c-b2b1-1b552c209562 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:53.883036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c3829cdc-34ed-4f7c-b2b1-1b552c209562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:53.907942 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:53.944606 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db97ac7-a2c3-4d94-a8e5-d50821b965cb admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:53.950345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2846db7-97c5-485c-9218-f601e1893087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:53.951024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2846db7-97c5-485c-9218-f601e1893087 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:53.951306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2846db7-97c5-485c-9218-f601e1893087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:53.951592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2846db7-97c5-485c-9218-f601e1893087 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:53.952547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2846db7-97c5-485c-9218-f601e1893087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:53.954504 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2846db7-97c5-485c-9218-f601e1893087 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:53.960110 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3829cdc-34ed-4f7c-b2b1-1b552c209562 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:53.966861 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55353e01-8d3f-4280-a300-0996aa7e2069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:53.967825 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55353e01-8d3f-4280-a300-0996aa7e2069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:53.968120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55353e01-8d3f-4280-a300-0996aa7e2069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:53.968430 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55353e01-8d3f-4280-a300-0996aa7e2069 admin admin] PATH_INFO: `/v3/users/bc15061650b94e8a969eb6c80112d04a` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:53.969858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55353e01-8d3f-4280-a300-0996aa7e2069 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc15061650b94e8a969eb6c80112d04a)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:53.971970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55353e01-8d3f-4280-a300-0996aa7e2069 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:53.982534 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1196/2428] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:36:53 2022] POST /identity/v3/users => generated 309 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:36:53.983720 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2846db7-97c5-485c-9218-f601e1893087 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:53.985439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e2846db7-97c5-485c-9218-f601e1893087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:53.998737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2846db7-97c5-485c-9218-f601e1893087 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:54.004161 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf82e8-f524-4264-a6db-22762ebdd897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:54.004731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:INSERT=2,keystone:SELECT=87,keystone:DELETE=11 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:54.005009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf82e8-f524-4264-a6db-22762ebdd897 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:54.005389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf82e8-f524-4264-a6db-22762ebdd897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:54.005841 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf82e8-f524-4264-a6db-22762ebdd897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:54.007021 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf82e8-f524-4264-a6db-22762ebdd897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:54.008631 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf82e8-f524-4264-a6db-22762ebdd897 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:54.021171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1197/2429] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:54.025574 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf82e8-f524-4264-a6db-22762ebdd897 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:54.034583 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-55353e01-8d3f-4280-a300-0996aa7e2069 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc15061650b94e8a969eb6c80112d04a'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:54.038578 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf82e8-f524-4264-a6db-22762ebdd897 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:54.044084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad4030-0d52-4a5d-8ee3-7e0b80f048b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:54.045323 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad4030-0d52-4a5d-8ee3-7e0b80f048b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:54.045706 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad4030-0d52-4a5d-8ee3-7e0b80f048b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:54.046202 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad4030-0d52-4a5d-8ee3-7e0b80f048b1 admin admin] PATH_INFO: `/v3/projects/1d721810efdc43b796e9bc0c9fedc40d/users/a1e396f657e346f9b6e0e33b5b36dc8d/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:54.048150 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-55353e01-8d3f-4280-a300-0996aa7e2069 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc15061650b94e8a969eb6c80112d04a'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:54.048931 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-55353e01-8d3f-4280-a300-0996aa7e2069 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc15061650b94e8a969eb6c80112d04a'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:54.060135 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-55353e01-8d3f-4280-a300-0996aa7e2069 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc15061650b94e8a969eb6c80112d04a'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:54.061582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-55353e01-8d3f-4280-a300-0996aa7e2069 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc15061650b94e8a969eb6c80112d04a'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:54.065306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad4030-0d52-4a5d-8ee3-7e0b80f048b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d721810efdc43b796e9bc0c9fedc40d, user_id=a1e396f657e346f9b6e0e33b5b36dc8d, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:54.068522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad4030-0d52-4a5d-8ee3-7e0b80f048b1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:54.079137 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1233/2430] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:36:53 2022] DELETE /identity/v3/users/bc15061650b94e8a969eb6c80112d04a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:54.091643 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1198/2431] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:54 2022] PUT /identity/v3/projects/1d721810efdc43b796e9bc0c9fedc40d/users/a1e396f657e346f9b6e0e33b5b36dc8d/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:54.095873 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55353e01-8d3f-4280-a300-0996aa7e2069 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:54.097581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-55353e01-8d3f-4280-a300-0996aa7e2069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:54.175522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55353e01-8d3f-4280-a300-0996aa7e2069 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:54.252597 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94031f1d-2428-4357-b9bb-625481ed0e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:54.253401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94031f1d-2428-4357-b9bb-625481ed0e65 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:54.253684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94031f1d-2428-4357-b9bb-625481ed0e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:54.253960 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94031f1d-2428-4357-b9bb-625481ed0e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:54.255077 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94031f1d-2428-4357-b9bb-625481ed0e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:54.256358 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94031f1d-2428-4357-b9bb-625481ed0e65 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:54.268360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1234/2432] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:54.274054 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad4030-0d52-4a5d-8ee3-7e0b80f048b1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:54.287878 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad4030-0d52-4a5d-8ee3-7e0b80f048b1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:54.293359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee64533-a37a-407e-a3a5-208d454feb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:54.294369 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee64533-a37a-407e-a3a5-208d454feb78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:54.294653 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee64533-a37a-407e-a3a5-208d454feb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:54.294945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee64533-a37a-407e-a3a5-208d454feb78 admin admin] PATH_INFO: `/v3/projects/1d721810efdc43b796e9bc0c9fedc40d/users/a1e396f657e346f9b6e0e33b5b36dc8d/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:54.296956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee64533-a37a-407e-a3a5-208d454feb78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d721810efdc43b796e9bc0c9fedc40d, user_id=a1e396f657e346f9b6e0e33b5b36dc8d, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:54.299654 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee64533-a37a-407e-a3a5-208d454feb78 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:54.311119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1199/2433] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:54 2022] PUT /identity/v3/projects/1d721810efdc43b796e9bc0c9fedc40d/users/a1e396f657e346f9b6e0e33b5b36dc8d/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:54.321557 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bf45f-e2b4-404c-811c-769596b28de6 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:54.321845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bf45f-e2b4-404c-811c-769596b28de6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:54.322123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bf45f-e2b4-404c-811c-769596b28de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:54.348812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-359bf45f-e2b4-404c-811c-769596b28de6 None None] MFA Rules not processed for user `a1e396f657e346f9b6e0e33b5b36dc8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:54.350412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-359bf45f-e2b4-404c-811c-769596b28de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:54.389917 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:54.430289 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1235/2434] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:36:54.441549 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee64533-a37a-407e-a3a5-208d454feb78 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:54.443243 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6ee64533-a37a-407e-a3a5-208d454feb78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:54.511223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee64533-a37a-407e-a3a5-208d454feb78 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:54.516013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ddd73-a6ce-412d-abdc-5ac09b24a89a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:54.516607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ddd73-a6ce-412d-abdc-5ac09b24a89a service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:54.516898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ddd73-a6ce-412d-abdc-5ac09b24a89a service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:54.517229 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ddd73-a6ce-412d-abdc-5ac09b24a89a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:54.530135 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ddd73-a6ce-412d-abdc-5ac09b24a89a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:54.531786 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ddd73-a6ce-412d-abdc-5ac09b24a89a service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:54.542152 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1200/2435] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:54 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:54.597472 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bf45f-e2b4-404c-811c-769596b28de6 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:54.599070 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-359bf45f-e2b4-404c-811c-769596b28de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:54.695419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bf45f-e2b4-404c-811c-769596b28de6 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:54.700238 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ddd73-a6ce-412d-abdc-5ac09b24a89a service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:54.704983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d89997-998d-4756-b98a-7cd396327e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:54.705801 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d89997-998d-4756-b98a-7cd396327e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:54.706083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d89997-998d-4756-b98a-7cd396327e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:54.706357 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d89997-998d-4756-b98a-7cd396327e8f admin admin] PATH_INFO: `/v3/projects/a26222767ad1431b89d65bcb96658a0b` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:54.707627 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d89997-998d-4756-b98a-7cd396327e8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a26222767ad1431b89d65bcb96658a0b)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:54.710033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d89997-998d-4756-b98a-7cd396327e8f admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:54.718095 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ddd73-a6ce-412d-abdc-5ac09b24a89a service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:54.724248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb41729-1051-405c-b0e5-3a00f9dba1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:54.725200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb41729-1051-405c-b0e5-3a00f9dba1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:54.725491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb41729-1051-405c-b0e5-3a00f9dba1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:54.725770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb41729-1051-405c-b0e5-3a00f9dba1a2 admin admin] PATH_INFO: `/v3/users/5a23b34c425f4764b342026a38b1ea07` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:54.726979 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb41729-1051-405c-b0e5-3a00f9dba1a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a23b34c425f4764b342026a38b1ea07)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:54.729064 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb41729-1051-405c-b0e5-3a00f9dba1a2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:54.784475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-59d89997-998d-4756-b98a-7cd396327e8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a26222767ad1431b89d65bcb96658a0b', 'request_id': 'req-59d89997-998d-4756-b98a-7cd396327e8f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:54.791237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-59d89997-998d-4756-b98a-7cd396327e8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a26222767ad1431b89d65bcb96658a0b', 'request_id': 'req-59d89997-998d-4756-b98a-7cd396327e8f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:54.799476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-59d89997-998d-4756-b98a-7cd396327e8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a26222767ad1431b89d65bcb96658a0b', 'request_id': 'req-59d89997-998d-4756-b98a-7cd396327e8f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:54.801926 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0bb41729-1051-405c-b0e5-3a00f9dba1a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a23b34c425f4764b342026a38b1ea07'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:54.815682 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-59d89997-998d-4756-b98a-7cd396327e8f admin admin] The token cache is being invalidate because project a26222767ad1431b89d65bcb96658a0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:36:54.815918 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0bb41729-1051-405c-b0e5-3a00f9dba1a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a23b34c425f4764b342026a38b1ea07'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:54.816031 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-59d89997-998d-4756-b98a-7cd396327e8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:54.816603 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0bb41729-1051-405c-b0e5-3a00f9dba1a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a23b34c425f4764b342026a38b1ea07'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:54.816724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-59d89997-998d-4756-b98a-7cd396327e8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:54.817474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0bb41729-1051-405c-b0e5-3a00f9dba1a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a23b34c425f4764b342026a38b1ea07'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:54.818223 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1236/2436] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:36:54 2022] DELETE /identity/v3/projects/a26222767ad1431b89d65bcb96658a0b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:54.823159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087baba-2db6-48e8-853e-9d11f728dbcb None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:54.823449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087baba-2db6-48e8-853e-9d11f728dbcb None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:54.824280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087baba-2db6-48e8-853e-9d11f728dbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:54.825351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:54.827617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0bb41729-1051-405c-b0e5-3a00f9dba1a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a23b34c425f4764b342026a38b1ea07'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:54.845084 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-f087baba-2db6-48e8-853e-9d11f728dbcb None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:54.848650 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1201/2437] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:36:54 2022] DELETE /identity/v3/users/5a23b34c425f4764b342026a38b1ea07 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:54.848791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-f087baba-2db6-48e8-853e-9d11f728dbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:54.850527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ad020-ac4c-4d6a-aecf-824950ed715f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:54.851050 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ad020-ac4c-4d6a-aecf-824950ed715f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:54.851905 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ad020-ac4c-4d6a-aecf-824950ed715f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:54.872214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-236ad020-ac4c-4d6a-aecf-824950ed715f None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:54.873695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-236ad020-ac4c-4d6a-aecf-824950ed715f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:54.917263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1237/2438] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:36:54.924881 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b008ec0-9c73-472a-89ab-9ee8386be381 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:54.925189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b008ec0-9c73-472a-89ab-9ee8386be381 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:54.925476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b008ec0-9c73-472a-89ab-9ee8386be381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:54.934035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:54.963195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-1b008ec0-9c73-472a-89ab-9ee8386be381 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:54.963967 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1202/2439] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:54.974082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1b008ec0-9c73-472a-89ab-9ee8386be381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:54.976738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ad020-ac4c-4d6a-aecf-824950ed715f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:54.977124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-236ad020-ac4c-4d6a-aecf-824950ed715f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:54.985182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1238/2440] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:36:54.990464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b008ec0-9c73-472a-89ab-9ee8386be381 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:55.002398 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b008ec0-9c73-472a-89ab-9ee8386be381 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:55.007279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be13153-0a61-4515-b7a1-5b98afeedc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:55.007949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be13153-0a61-4515-b7a1-5b98afeedc11 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:55.008236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be13153-0a61-4515-b7a1-5b98afeedc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:55.008514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be13153-0a61-4515-b7a1-5b98afeedc11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:55.009694 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be13153-0a61-4515-b7a1-5b98afeedc11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:36:55.010006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be13153-0a61-4515-b7a1-5b98afeedc11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:55.012904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be13153-0a61-4515-b7a1-5b98afeedc11 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:55.026298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1239/2441] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:36:55.031396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be13153-0a61-4515-b7a1-5b98afeedc11 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:55.041926 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be13153-0a61-4515-b7a1-5b98afeedc11 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:55.045510 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ad020-ac4c-4d6a-aecf-824950ed715f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:55.048718 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c062f3-78a6-4cf3-b581-f2a4d8ac4d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:55.049494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c062f3-78a6-4cf3-b581-f2a4d8ac4d7b admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:55.049853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c062f3-78a6-4cf3-b581-f2a4d8ac4d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:55.050201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c062f3-78a6-4cf3-b581-f2a4d8ac4d7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:55.050461 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c31ff-ffeb-43ce-a667-a12a99f9e6cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:55.051033 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c062f3-78a6-4cf3-b581-f2a4d8ac4d7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:55.051704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c31ff-ffeb-43ce-a667-a12a99f9e6cc service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:55.052150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c31ff-ffeb-43ce-a667-a12a99f9e6cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:55.052591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c31ff-ffeb-43ce-a667-a12a99f9e6cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:55.053429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c062f3-78a6-4cf3-b581-f2a4d8ac4d7b admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:55.076216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c31ff-ffeb-43ce-a667-a12a99f9e6cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:55.078738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c31ff-ffeb-43ce-a667-a12a99f9e6cc service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:55.087493 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1240/2442] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:36:55 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:36:55.088455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c062f3-78a6-4cf3-b581-f2a4d8ac4d7b admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:55.096985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1203/2443] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:36:55.195795 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c062f3-78a6-4cf3-b581-f2a4d8ac4d7b admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:55.205048 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf35455-d6f1-46bc-a88b-2e1e7c45b6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:55.214134 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf35455-d6f1-46bc-a88b-2e1e7c45b6f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:55.214925 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf35455-d6f1-46bc-a88b-2e1e7c45b6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:55.215141 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf35455-d6f1-46bc-a88b-2e1e7c45b6f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:55.215861 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf35455-d6f1-46bc-a88b-2e1e7c45b6f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:55.217884 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf35455-d6f1-46bc-a88b-2e1e7c45b6f0 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:55.244894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1241/2444] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:36:55 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:36:55.252614 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c31ff-ffeb-43ce-a667-a12a99f9e6cc service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:55.266525 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c31ff-ffeb-43ce-a667-a12a99f9e6cc service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:55.280786 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccb5f8-5728-4674-a1f3-7826467055f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:55.281579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccb5f8-5728-4674-a1f3-7826467055f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:55.281871 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccb5f8-5728-4674-a1f3-7826467055f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:55.282150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccb5f8-5728-4674-a1f3-7826467055f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:55.282947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccb5f8-5728-4674-a1f3-7826467055f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:55.285336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccb5f8-5728-4674-a1f3-7826467055f9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:55.318376 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1204/2445] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:55.323288 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf35455-d6f1-46bc-a88b-2e1e7c45b6f0 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:55.335280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf35455-d6f1-46bc-a88b-2e1e7c45b6f0 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:55.340810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031cdba-a128-47c2-a4dc-63733c39e7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:55.341369 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031cdba-a128-47c2-a4dc-63733c39e7ab admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:55.341830 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031cdba-a128-47c2-a4dc-63733c39e7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:55.342083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031cdba-a128-47c2-a4dc-63733c39e7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:55.342823 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031cdba-a128-47c2-a4dc-63733c39e7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:55.344676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031cdba-a128-47c2-a4dc-63733c39e7ab admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:55.364963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1242/2446] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:55.365546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccb5f8-5728-4674-a1f3-7826467055f9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:55.376956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccb5f8-5728-4674-a1f3-7826467055f9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:55.382990 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a3f88-2939-434c-bda3-0e5a63f4c9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:55.383914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a3f88-2939-434c-bda3-0e5a63f4c9a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:55.384194 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a3f88-2939-434c-bda3-0e5a63f4c9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:55.384465 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a3f88-2939-434c-bda3-0e5a63f4c9a0 admin admin] PATH_INFO: `/v3/projects/6f6f109d541a41448a1085a7f469339f/users/3f7c5f3286ee4593a19885d175f51b00/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:55.404747 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a3f88-2939-434c-bda3-0e5a63f4c9a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6f109d541a41448a1085a7f469339f, user_id=3f7c5f3286ee4593a19885d175f51b00, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:55.407868 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a3f88-2939-434c-bda3-0e5a63f4c9a0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:55.430485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1205/2447] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:55 2022] PUT /identity/v3/projects/6f6f109d541a41448a1085a7f469339f/users/3f7c5f3286ee4593a19885d175f51b00/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:55.443998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c782c3c-5083-4537-8fd8-73633893c258 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:55.444291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c782c3c-5083-4537-8fd8-73633893c258 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:55.444644 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c782c3c-5083-4537-8fd8-73633893c258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:55.480625 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-4c782c3c-5083-4537-8fd8-73633893c258 None None] MFA Rules not processed for user `3f7c5f3286ee4593a19885d175f51b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:55.482211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-4c782c3c-5083-4537-8fd8-73633893c258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:55.535845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a3f88-2939-434c-bda3-0e5a63f4c9a0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:55.538965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f61a3f88-2939-434c-bda3-0e5a63f4c9a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:55.618071 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1243/2448] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:55.619522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:55.623321 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c782c3c-5083-4537-8fd8-73633893c258 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:55.624704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a3f88-2939-434c-bda3-0e5a63f4c9a0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:55.633418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:55.634838 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:55.635718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:55.637741 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393 admin admin] PATH_INFO: `/v3/projects/6046006ba4f3451694b4ebe24a076a27` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:55.639130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6046006ba4f3451694b4ebe24a076a27)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:55.641821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:55.642577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c782c3c-5083-4537-8fd8-73633893c258 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:55.648501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e17fe-2360-4ca2-95e2-da05282ac395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:55.649288 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e17fe-2360-4ca2-95e2-da05282ac395 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:55.649569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e17fe-2360-4ca2-95e2-da05282ac395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:55.649847 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e17fe-2360-4ca2-95e2-da05282ac395 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:55.650539 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e17fe-2360-4ca2-95e2-da05282ac395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:55.654056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e17fe-2360-4ca2-95e2-da05282ac395 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:55.687381 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1244/2449] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:36:55 2022] POST /identity/v3/projects => generated 387 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:36:55.691535 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e17fe-2360-4ca2-95e2-da05282ac395 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:55.703096 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:55.706564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6046006ba4f3451694b4ebe24a076a27', 'request_id': 'req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:55.717611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6046006ba4f3451694b4ebe24a076a27', 'request_id': 'req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:55.725434 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6046006ba4f3451694b4ebe24a076a27', 'request_id': 'req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:55.740331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393 admin admin] The token cache is being invalidate because project 6046006ba4f3451694b4ebe24a076a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:36:55.740660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:55.743364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:55.747387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1206/2450] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:36:55 2022] DELETE /identity/v3/projects/6046006ba4f3451694b4ebe24a076a27 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:55.748561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:55.750024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:55.775426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e17fe-2360-4ca2-95e2-da05282ac395 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:55.832029 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d27af-d1d2-46f9-91f2-81e7e8a7b393 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:55.839780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ec19c-27d4-4d8c-82af-a08f1684e368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:55.840775 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ec19c-27d4-4d8c-82af-a08f1684e368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:55.841226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ec19c-27d4-4d8c-82af-a08f1684e368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:55.841604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ec19c-27d4-4d8c-82af-a08f1684e368 admin admin] PATH_INFO: `/v3/users/a1e396f657e346f9b6e0e33b5b36dc8d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:55.842967 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ec19c-27d4-4d8c-82af-a08f1684e368 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1e396f657e346f9b6e0e33b5b36dc8d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:55.845040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ec19c-27d4-4d8c-82af-a08f1684e368 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:55.860761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf189d7-f0c8-44a1-b53b-98b3375ac7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:55.862408 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf189d7-f0c8-44a1-b53b-98b3375ac7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:55.862855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf189d7-f0c8-44a1-b53b-98b3375ac7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:55.863586 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf189d7-f0c8-44a1-b53b-98b3375ac7a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:55.864716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf189d7-f0c8-44a1-b53b-98b3375ac7a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:55.867853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf189d7-f0c8-44a1-b53b-98b3375ac7a9 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:55.896642 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1245/2451] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:36:55 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:36:55.903742 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf189d7-f0c8-44a1-b53b-98b3375ac7a9 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:55.906145 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1cf189d7-f0c8-44a1-b53b-98b3375ac7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:55.919426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf189d7-f0c8-44a1-b53b-98b3375ac7a9 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:55.925663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22a8f8-c2f4-427b-9703-265b006ead79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:55.926480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22a8f8-c2f4-427b-9703-265b006ead79 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:55.926766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22a8f8-c2f4-427b-9703-265b006ead79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:55.927045 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22a8f8-c2f4-427b-9703-265b006ead79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:55.927781 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22a8f8-c2f4-427b-9703-265b006ead79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:55.929514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22a8f8-c2f4-427b-9703-265b006ead79 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:55.943427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-752ec19c-27d4-4d8c-82af-a08f1684e368 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1e396f657e346f9b6e0e33b5b36dc8d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:55.948216 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1246/2452] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:55.953464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22a8f8-c2f4-427b-9703-265b006ead79 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:55.970991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-752ec19c-27d4-4d8c-82af-a08f1684e368 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1e396f657e346f9b6e0e33b5b36dc8d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:55.972423 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-752ec19c-27d4-4d8c-82af-a08f1684e368 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1e396f657e346f9b6e0e33b5b36dc8d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:55.973336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-752ec19c-27d4-4d8c-82af-a08f1684e368 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1e396f657e346f9b6e0e33b5b36dc8d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:55.977339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22a8f8-c2f4-427b-9703-265b006ead79 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:55.986490 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-752ec19c-27d4-4d8c-82af-a08f1684e368 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1e396f657e346f9b6e0e33b5b36dc8d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:55.987769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b3f33-4b2a-4395-a5b1-0210ed67a0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:55.988251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b3f33-4b2a-4395-a5b1-0210ed67a0be admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:55.988529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b3f33-4b2a-4395-a5b1-0210ed67a0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:55.988958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b3f33-4b2a-4395-a5b1-0210ed67a0be admin admin] PATH_INFO: `/v3/projects/b16f62df8b004961b85bbdb454a3ee37/users/073159ba5fa14fda8031c264cc17900a/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:56.015983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1207/2453] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:36:55 2022] DELETE /identity/v3/users/a1e396f657e346f9b6e0e33b5b36dc8d => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:56.016329 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b3f33-4b2a-4395-a5b1-0210ed67a0be admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16f62df8b004961b85bbdb454a3ee37, user_id=073159ba5fa14fda8031c264cc17900a, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:56.016329 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b3f33-4b2a-4395-a5b1-0210ed67a0be admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:56.047177 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1247/2454] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:55 2022] PUT /identity/v3/projects/b16f62df8b004961b85bbdb454a3ee37/users/073159ba5fa14fda8031c264cc17900a/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:56.051006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ec19c-27d4-4d8c-82af-a08f1684e368 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:56.054216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-752ec19c-27d4-4d8c-82af-a08f1684e368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:56.077404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:56.225748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ec19c-27d4-4d8c-82af-a08f1684e368 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:56.285969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055b970-8e77-45ad-b34f-a7292f22b097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:56.286652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055b970-8e77-45ad-b34f-a7292f22b097 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:56.287015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055b970-8e77-45ad-b34f-a7292f22b097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:56.287302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055b970-8e77-45ad-b34f-a7292f22b097 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:56.288085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055b970-8e77-45ad-b34f-a7292f22b097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:56.289740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055b970-8e77-45ad-b34f-a7292f22b097 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:56.300586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1208/2455] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:36:56.304817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b3f33-4b2a-4395-a5b1-0210ed67a0be admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:56.315431 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b3f33-4b2a-4395-a5b1-0210ed67a0be admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:56.320200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdd2f7-6e96-4b15-8b31-41eeb997584d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:56.321099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdd2f7-6e96-4b15-8b31-41eeb997584d admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:56.321405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdd2f7-6e96-4b15-8b31-41eeb997584d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:56.321679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdd2f7-6e96-4b15-8b31-41eeb997584d admin admin] PATH_INFO: `/v3/projects/b16f62df8b004961b85bbdb454a3ee37/users/073159ba5fa14fda8031c264cc17900a/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:56.324799 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bdd2f7-6e96-4b15-8b31-41eeb997584d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16f62df8b004961b85bbdb454a3ee37, user_id=073159ba5fa14fda8031c264cc17900a, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:56.327486 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bdd2f7-6e96-4b15-8b31-41eeb997584d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:56.341556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1248/2456] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:56 2022] PUT /identity/v3/projects/b16f62df8b004961b85bbdb454a3ee37/users/073159ba5fa14fda8031c264cc17900a/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:56.351606 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca19bc-f888-4e53-b6d1-13d3ff3881af None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:56.351898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca19bc-f888-4e53-b6d1-13d3ff3881af None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:56.352180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca19bc-f888-4e53-b6d1-13d3ff3881af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:56.386808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b6ca19bc-f888-4e53-b6d1-13d3ff3881af None None] MFA Rules not processed for user `073159ba5fa14fda8031c264cc17900a`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:56.388255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b6ca19bc-f888-4e53-b6d1-13d3ff3881af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:56.425319 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdd2f7-6e96-4b15-8b31-41eeb997584d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:56.427539 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b7bdd2f7-6e96-4b15-8b31-41eeb997584d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:56.499550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1209/2457] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:56.514488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca19bc-f888-4e53-b6d1-13d3ff3881af None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:56.516186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b6ca19bc-f888-4e53-b6d1-13d3ff3881af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:56.553595 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:56.562536 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdd2f7-6e96-4b15-8b31-41eeb997584d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:56.566144 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:56.570806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b2881-0548-4f79-9718-a11427dfb583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:56.572476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b2881-0548-4f79-9718-a11427dfb583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:56.573593 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b2881-0548-4f79-9718-a11427dfb583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:56.574060 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b2881-0548-4f79-9718-a11427dfb583 admin admin] PATH_INFO: `/v3/projects/1d721810efdc43b796e9bc0c9fedc40d` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:56.576360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b2881-0548-4f79-9718-a11427dfb583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d721810efdc43b796e9bc0c9fedc40d)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:56.578726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b2881-0548-4f79-9718-a11427dfb583 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:56.588446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca19bc-f888-4e53-b6d1-13d3ff3881af None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:56.595503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84c9dc-1ef1-47d0-8f6f-30c0e3a01b8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:56.596261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84c9dc-1ef1-47d0-8f6f-30c0e3a01b8b service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:56.596897 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84c9dc-1ef1-47d0-8f6f-30c0e3a01b8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:56.597489 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84c9dc-1ef1-47d0-8f6f-30c0e3a01b8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:56.613557 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84c9dc-1ef1-47d0-8f6f-30c0e3a01b8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:56.616173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84c9dc-1ef1-47d0-8f6f-30c0e3a01b8b service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:56.641483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1210/2458] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:56.653950 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-720b2881-0548-4f79-9718-a11427dfb583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d721810efdc43b796e9bc0c9fedc40d', 'request_id': 'req-720b2881-0548-4f79-9718-a11427dfb583'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:56.671333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-720b2881-0548-4f79-9718-a11427dfb583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d721810efdc43b796e9bc0c9fedc40d', 'request_id': 'req-720b2881-0548-4f79-9718-a11427dfb583'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:56.679701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-720b2881-0548-4f79-9718-a11427dfb583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d721810efdc43b796e9bc0c9fedc40d', 'request_id': 'req-720b2881-0548-4f79-9718-a11427dfb583'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:56.697814 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-720b2881-0548-4f79-9718-a11427dfb583 admin admin] The token cache is being invalidate because project 1d721810efdc43b796e9bc0c9fedc40d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:36:56.698111 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-720b2881-0548-4f79-9718-a11427dfb583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:56.699574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-720b2881-0548-4f79-9718-a11427dfb583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:56.701285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84c9dc-1ef1-47d0-8f6f-30c0e3a01b8b service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:56.702188 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1249/2459] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:36:56 2022] DELETE /identity/v3/projects/1d721810efdc43b796e9bc0c9fedc40d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:56.708293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a98ec-746d-4678-b13f-865549ae3c6c None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:56.708698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a98ec-746d-4678-b13f-865549ae3c6c None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:56.709405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a98ec-746d-4678-b13f-865549ae3c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:56.713039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84c9dc-1ef1-47d0-8f6f-30c0e3a01b8b service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:56.735171 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ea1a98ec-746d-4678-b13f-865549ae3c6c None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:56.736416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ea1a98ec-746d-4678-b13f-865549ae3c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:56.781608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40648c1-5897-4268-a24a-58d17a3615e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:56.782877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40648c1-5897-4268-a24a-58d17a3615e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:56.783165 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40648c1-5897-4268-a24a-58d17a3615e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:56.783758 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40648c1-5897-4268-a24a-58d17a3615e0 admin admin] PATH_INFO: `/v3/users/3f7c5f3286ee4593a19885d175f51b00` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:56.785350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40648c1-5897-4268-a24a-58d17a3615e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f7c5f3286ee4593a19885d175f51b00)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:56.787680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40648c1-5897-4268-a24a-58d17a3615e0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:56.803851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1250/2460] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:36:56.811549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee5cb0-aa77-41d2-8afd-dbd46b50186a None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:56.811850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee5cb0-aa77-41d2-8afd-dbd46b50186a None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:56.812151 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee5cb0-aa77-41d2-8afd-dbd46b50186a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:56.832095 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-9cee5cb0-aa77-41d2-8afd-dbd46b50186a None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:56.833213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9cee5cb0-aa77-41d2-8afd-dbd46b50186a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:56.845185 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1251/2461] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:56.846719 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee5cb0-aa77-41d2-8afd-dbd46b50186a None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:56.857254 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee5cb0-aa77-41d2-8afd-dbd46b50186a None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:56.862603 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715316c-c4e4-439a-ae2a-2f300f08a08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:56.863264 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715316c-c4e4-439a-ae2a-2f300f08a08c admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:56.863417 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c40648c1-5897-4268-a24a-58d17a3615e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f7c5f3286ee4593a19885d175f51b00'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:56.863552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715316c-c4e4-439a-ae2a-2f300f08a08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:56.863920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715316c-c4e4-439a-ae2a-2f300f08a08c admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:56.864781 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715316c-c4e4-439a-ae2a-2f300f08a08c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:36:56.865092 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715316c-c4e4-439a-ae2a-2f300f08a08c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:56.866880 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715316c-c4e4-439a-ae2a-2f300f08a08c admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:56.878944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c40648c1-5897-4268-a24a-58d17a3615e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f7c5f3286ee4593a19885d175f51b00'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:56.879800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c40648c1-5897-4268-a24a-58d17a3615e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f7c5f3286ee4593a19885d175f51b00'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:56.880604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c40648c1-5897-4268-a24a-58d17a3615e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f7c5f3286ee4593a19885d175f51b00'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:56.881001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1252/2462] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:36:56.887675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715316c-c4e4-439a-ae2a-2f300f08a08c admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:56.889458 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7715316c-c4e4-439a-ae2a-2f300f08a08c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:56.889770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c40648c1-5897-4268-a24a-58d17a3615e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f7c5f3286ee4593a19885d175f51b00'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:56.903596 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715316c-c4e4-439a-ae2a-2f300f08a08c admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:56.907948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1211/2463] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:36:56 2022] DELETE /identity/v3/users/3f7c5f3286ee4593a19885d175f51b00 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:56.908828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f5a69-61b6-4a1e-b9d8-045db200a1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:56.909545 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f5a69-61b6-4a1e-b9d8-045db200a1e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:56.909830 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f5a69-61b6-4a1e-b9d8-045db200a1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:56.910725 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f5a69-61b6-4a1e-b9d8-045db200a1e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:56.910958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f5a69-61b6-4a1e-b9d8-045db200a1e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:56.917951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f5a69-61b6-4a1e-b9d8-045db200a1e1 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:56.920488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5009c-5eb5-42c4-8d20-9129896ddfbe None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:56.920788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5009c-5eb5-42c4-8d20-9129896ddfbe None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:56.921109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5009c-5eb5-42c4-8d20-9129896ddfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:56.941653 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bce5009c-5eb5-42c4-8d20-9129896ddfbe None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:56.943026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bce5009c-5eb5-42c4-8d20-9129896ddfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:56.951099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1253/2464] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:36:56 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:36:56.954643 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f5a69-61b6-4a1e-b9d8-045db200a1e1 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:57.005044 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:57.010846 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:57.019854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1212/2465] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:57.030453 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5009c-5eb5-42c4-8d20-9129896ddfbe None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:57.032319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bce5009c-5eb5-42c4-8d20-9129896ddfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:57.034932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f5a69-61b6-4a1e-b9d8-045db200a1e1 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:57.040042 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a7f59-e74a-459e-9656-20477006e3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:57.040779 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a7f59-e74a-459e-9656-20477006e3a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.041069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a7f59-e74a-459e-9656-20477006e3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.041381 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a7f59-e74a-459e-9656-20477006e3a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:57.042118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a7f59-e74a-459e-9656-20477006e3a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:57.044134 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a7f59-e74a-459e-9656-20477006e3a2 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:57.072442 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1254/2466] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:36:56 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:36:57.075132 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a7f59-e74a-459e-9656-20477006e3a2 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:57.090011 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a7f59-e74a-459e-9656-20477006e3a2 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:57.095088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c34968-2814-4297-bb50-7c81d0e93bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:57.095785 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c34968-2814-4297-bb50-7c81d0e93bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.096066 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c34968-2814-4297-bb50-7c81d0e93bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.096396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c34968-2814-4297-bb50-7c81d0e93bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:57.097294 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c34968-2814-4297-bb50-7c81d0e93bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:57.099091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c34968-2814-4297-bb50-7c81d0e93bb1 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:57.108720 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5009c-5eb5-42c4-8d20-9129896ddfbe None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:57.117450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1255/2467] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:57.122150 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c34968-2814-4297-bb50-7c81d0e93bb1 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:57.132032 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c34968-2814-4297-bb50-7c81d0e93bb1 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:57.137467 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4c9ba-dde3-4032-901a-f75217574200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:57.138119 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4c9ba-dde3-4032-901a-f75217574200 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.138406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4c9ba-dde3-4032-901a-f75217574200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.138824 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4c9ba-dde3-4032-901a-f75217574200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:57.139583 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4c9ba-dde3-4032-901a-f75217574200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:57.141206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4c9ba-dde3-4032-901a-f75217574200 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:57.152869 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1256/2468] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:57.156866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4c9ba-dde3-4032-901a-f75217574200 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:57.162812 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fff4ba-8221-47af-8ecf-62913f7943b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:57.163469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fff4ba-8221-47af-8ecf-62913f7943b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.163753 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fff4ba-8221-47af-8ecf-62913f7943b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.164034 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fff4ba-8221-47af-8ecf-62913f7943b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:57.166955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4c9ba-dde3-4032-901a-f75217574200 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:57.171736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7c04a-a828-43cb-b36a-029a805ef0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:57.172599 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7c04a-a828-43cb-b36a-029a805ef0d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.172879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7c04a-a828-43cb-b36a-029a805ef0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.173192 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7c04a-a828-43cb-b36a-029a805ef0d7 admin admin] PATH_INFO: `/v3/projects/27a47546a133461baa7e1bead1810410/users/7f3f3412f42142ef9562199733dbfc6f/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:57.173750 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fff4ba-8221-47af-8ecf-62913f7943b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:57.175181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fff4ba-8221-47af-8ecf-62913f7943b9 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:57.185552 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1213/2469] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:36:57.189559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7c04a-a828-43cb-b36a-029a805ef0d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27a47546a133461baa7e1bead1810410, user_id=7f3f3412f42142ef9562199733dbfc6f, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:57.192397 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7c04a-a828-43cb-b36a-029a805ef0d7 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:57.209178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1257/2470] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:57 2022] PUT /identity/v3/projects/27a47546a133461baa7e1bead1810410/users/7f3f3412f42142ef9562199733dbfc6f/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:57.219182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a6ee0-e028-4c68-83d4-8c6f1c9a8ad6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.219476 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a6ee0-e028-4c68-83d4-8c6f1c9a8ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.219928 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a6ee0-e028-4c68-83d4-8c6f1c9a8ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:57.250406 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-227a6ee0-e028-4c68-83d4-8c6f1c9a8ad6 None None] MFA Rules not processed for user `7f3f3412f42142ef9562199733dbfc6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:57.252018 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-227a6ee0-e028-4c68-83d4-8c6f1c9a8ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:57.331399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1214/2471] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:57.336035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7c04a-a828-43cb-b36a-029a805ef0d7 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:57.403708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7c04a-a828-43cb-b36a-029a805ef0d7 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:57.408704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d216004-1e26-4ecf-b0f9-f188d79d94f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:57.409411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d216004-1e26-4ecf-b0f9-f188d79d94f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.409701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d216004-1e26-4ecf-b0f9-f188d79d94f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.409977 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d216004-1e26-4ecf-b0f9-f188d79d94f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:57.410684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d216004-1e26-4ecf-b0f9-f188d79d94f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:57.412639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d216004-1e26-4ecf-b0f9-f188d79d94f5 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:57.440169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1258/2472] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:36:57 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:36:57.444298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a6ee0-e028-4c68-83d4-8c6f1c9a8ad6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:57.456805 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a6ee0-e028-4c68-83d4-8c6f1c9a8ad6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:57.482933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d216004-1e26-4ecf-b0f9-f188d79d94f5 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:57.484662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-4d216004-1e26-4ecf-b0f9-f188d79d94f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:57.505975 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:57.523598 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2311803-9f84-455f-bdb8-01d2e851ac14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:57.524279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2311803-9f84-455f-bdb8-01d2e851ac14 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.524562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2311803-9f84-455f-bdb8-01d2e851ac14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.524862 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2311803-9f84-455f-bdb8-01d2e851ac14 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:57.525591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2311803-9f84-455f-bdb8-01d2e851ac14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:57.527828 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2311803-9f84-455f-bdb8-01d2e851ac14 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:57.548949 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1215/2473] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:36:57 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:36:57.552675 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2311803-9f84-455f-bdb8-01d2e851ac14 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:57.563339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2311803-9f84-455f-bdb8-01d2e851ac14 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:57.565660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d216004-1e26-4ecf-b0f9-f188d79d94f5 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:57.568444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80258f-13fd-4fed-a3b1-e8ce9d25bd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:57.569102 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80258f-13fd-4fed-a3b1-e8ce9d25bd0d admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.569424 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80258f-13fd-4fed-a3b1-e8ce9d25bd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.569715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80258f-13fd-4fed-a3b1-e8ce9d25bd0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:57.570447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80258f-13fd-4fed-a3b1-e8ce9d25bd0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:57.571411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:57.572037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80258f-13fd-4fed-a3b1-e8ce9d25bd0d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:57.572409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.572771 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.573113 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf admin admin] PATH_INFO: `/v3/projects/6f6f109d541a41448a1085a7f469339f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:57.575050 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f6f109d541a41448a1085a7f469339f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:57.576989 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:57.583645 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1216/2474] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:36:57.587270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80258f-13fd-4fed-a3b1-e8ce9d25bd0d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:57.596361 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:57.597116 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80258f-13fd-4fed-a3b1-e8ce9d25bd0d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:57.601869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d3a2d-83cd-4028-9870-1b9999723520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:57.602729 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d3a2d-83cd-4028-9870-1b9999723520 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.603019 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d3a2d-83cd-4028-9870-1b9999723520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.603307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d3a2d-83cd-4028-9870-1b9999723520 admin admin] PATH_INFO: `/v3/projects/73d34046acab4b98a834a66ae712a8f3/users/0e00f0f609024462a31718a1c4173eb9/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:57.621221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d3a2d-83cd-4028-9870-1b9999723520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73d34046acab4b98a834a66ae712a8f3, user_id=0e00f0f609024462a31718a1c4173eb9, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:57.624033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d3a2d-83cd-4028-9870-1b9999723520 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:57.628734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f6f109d541a41448a1085a7f469339f', 'request_id': 'req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:57.635423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f6f109d541a41448a1085a7f469339f', 'request_id': 'req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:57.637390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1217/2475] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:57 2022] PUT /identity/v3/projects/73d34046acab4b98a834a66ae712a8f3/users/0e00f0f609024462a31718a1c4173eb9/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:57.641429 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d3a2d-83cd-4028-9870-1b9999723520 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:57.643924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f6f109d541a41448a1085a7f469339f', 'request_id': 'req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:57.656813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf admin admin] The token cache is being invalidate because project 6f6f109d541a41448a1085a7f469339f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:36:57.657109 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:57.657779 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:57.659337 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1259/2476] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:36:57 2022] DELETE /identity/v3/projects/6f6f109d541a41448a1085a7f469339f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:57.663239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:57.664567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:57.704217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d3a2d-83cd-4028-9870-1b9999723520 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:57.709176 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85a14e-9110-4f12-804a-956bc8e2941e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:57.709854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85a14e-9110-4f12-804a-956bc8e2941e admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.710143 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85a14e-9110-4f12-804a-956bc8e2941e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.710432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85a14e-9110-4f12-804a-956bc8e2941e admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:57.711158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85a14e-9110-4f12-804a-956bc8e2941e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:57.712762 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85a14e-9110-4f12-804a-956bc8e2941e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:57.722708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1218/2477] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:36:57.726766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85a14e-9110-4f12-804a-956bc8e2941e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:57.728413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cb85a14e-9110-4f12-804a-956bc8e2941e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:57.728532 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e6ba2-1666-4c57-a9c0-f20ae5615caf admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:57.733773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368a46b-7688-46e4-8715-52d80e4b84a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:57.734648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368a46b-7688-46e4-8715-52d80e4b84a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.734932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368a46b-7688-46e4-8715-52d80e4b84a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.735210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368a46b-7688-46e4-8715-52d80e4b84a2 admin admin] PATH_INFO: `/v3/users/073159ba5fa14fda8031c264cc17900a` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:57.736421 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368a46b-7688-46e4-8715-52d80e4b84a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=073159ba5fa14fda8031c264cc17900a)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:57.738387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368a46b-7688-46e4-8715-52d80e4b84a2 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:57.741538 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85a14e-9110-4f12-804a-956bc8e2941e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:57.747000 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b216b-f172-4d65-a8a1-62765e4a5734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:57.747871 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b216b-f172-4d65-a8a1-62765e4a5734 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.748160 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b216b-f172-4d65-a8a1-62765e4a5734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.748436 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b216b-f172-4d65-a8a1-62765e4a5734 admin admin] PATH_INFO: `/v3/projects/73d34046acab4b98a834a66ae712a8f3/users/0e00f0f609024462a31718a1c4173eb9/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:57.750437 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b216b-f172-4d65-a8a1-62765e4a5734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73d34046acab4b98a834a66ae712a8f3, user_id=0e00f0f609024462a31718a1c4173eb9, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:57.753087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b216b-f172-4d65-a8a1-62765e4a5734 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:57.763980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1219/2478] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:57 2022] PUT /identity/v3/projects/73d34046acab4b98a834a66ae712a8f3/users/0e00f0f609024462a31718a1c4173eb9/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:57.775314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cf10b-3d18-42ce-8535-560592515e5c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.775668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cf10b-3d18-42ce-8535-560592515e5c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.776038 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cf10b-3d18-42ce-8535-560592515e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:57.793410 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-0368a46b-7688-46e4-8715-52d80e4b84a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '073159ba5fa14fda8031c264cc17900a'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:57.804260 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-524cf10b-3d18-42ce-8535-560592515e5c None None] MFA Rules not processed for user `0e00f0f609024462a31718a1c4173eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:57.805528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-524cf10b-3d18-42ce-8535-560592515e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:57.805873 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-0368a46b-7688-46e4-8715-52d80e4b84a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '073159ba5fa14fda8031c264cc17900a'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:57.806544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-0368a46b-7688-46e4-8715-52d80e4b84a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '073159ba5fa14fda8031c264cc17900a'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:57.814757 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-0368a46b-7688-46e4-8715-52d80e4b84a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '073159ba5fa14fda8031c264cc17900a'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:57.815390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-0368a46b-7688-46e4-8715-52d80e4b84a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '073159ba5fa14fda8031c264cc17900a'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:57.828207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1260/2479] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:36:57 2022] DELETE /identity/v3/users/073159ba5fa14fda8031c264cc17900a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:57.876929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1220/2480] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:57 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:57.888579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368a46b-7688-46e4-8715-52d80e4b84a2 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:57.890678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0368a46b-7688-46e4-8715-52d80e4b84a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:57.919039 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:57.953451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368a46b-7688-46e4-8715-52d80e4b84a2 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:57.957234 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82899cd-baec-46bb-9384-30658f9d8e81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:57.957961 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82899cd-baec-46bb-9384-30658f9d8e81 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:57.958312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82899cd-baec-46bb-9384-30658f9d8e81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:57.958601 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82899cd-baec-46bb-9384-30658f9d8e81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:58.024383 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82899cd-baec-46bb-9384-30658f9d8e81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:58.025768 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82899cd-baec-46bb-9384-30658f9d8e81 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:58.036734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1261/2481] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:57 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:58.115009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cf10b-3d18-42ce-8535-560592515e5c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:58.116822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-524cf10b-3d18-42ce-8535-560592515e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:58.153605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:58.251014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cf10b-3d18-42ce-8535-560592515e5c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:58.339206 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4189d-ecf1-415f-9b84-3642e4c849fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:58.340108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4189d-ecf1-415f-9b84-3642e4c849fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:58.340399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4189d-ecf1-415f-9b84-3642e4c849fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:58.340687 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4189d-ecf1-415f-9b84-3642e4c849fc admin admin] PATH_INFO: `/v3/projects/b16f62df8b004961b85bbdb454a3ee37` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:58.342142 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4189d-ecf1-415f-9b84-3642e4c849fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b16f62df8b004961b85bbdb454a3ee37)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:58.345324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4189d-ecf1-415f-9b84-3642e4c849fc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:58.398428 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c4d4189d-ecf1-415f-9b84-3642e4c849fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b16f62df8b004961b85bbdb454a3ee37', 'request_id': 'req-c4d4189d-ecf1-415f-9b84-3642e4c849fc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:58.410721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c4d4189d-ecf1-415f-9b84-3642e4c849fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b16f62df8b004961b85bbdb454a3ee37', 'request_id': 'req-c4d4189d-ecf1-415f-9b84-3642e4c849fc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:58.417108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c4d4189d-ecf1-415f-9b84-3642e4c849fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b16f62df8b004961b85bbdb454a3ee37', 'request_id': 'req-c4d4189d-ecf1-415f-9b84-3642e4c849fc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:58.431219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c4d4189d-ecf1-415f-9b84-3642e4c849fc admin admin] The token cache is being invalidate because project b16f62df8b004961b85bbdb454a3ee37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:36:58.431637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c4d4189d-ecf1-415f-9b84-3642e4c849fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:58.432375 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c4d4189d-ecf1-415f-9b84-3642e4c849fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:58.434688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1221/2482] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:36:58 2022] DELETE /identity/v3/projects/b16f62df8b004961b85bbdb454a3ee37 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:58.442892 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40bb7c-9fdc-4307-a21c-ae21a4248939 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:58.443390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40bb7c-9fdc-4307-a21c-ae21a4248939 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:58.443738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40bb7c-9fdc-4307-a21c-ae21a4248939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:58.463683 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-eb40bb7c-9fdc-4307-a21c-ae21a4248939 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:58.470397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4189d-ecf1-415f-9b84-3642e4c849fc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:58.471351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-eb40bb7c-9fdc-4307-a21c-ae21a4248939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:58.471849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c4d4189d-ecf1-415f-9b84-3642e4c849fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:58.527660 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:58.540217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1262/2483] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:58.545904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4189d-ecf1-415f-9b84-3642e4c849fc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:58.548742 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bc116-a2a7-4afc-b6b1-e970d5cb7b18 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:58.549045 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bc116-a2a7-4afc-b6b1-e970d5cb7b18 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:58.549367 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bc116-a2a7-4afc-b6b1-e970d5cb7b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:58.551051 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7c52a-a509-45d1-ba26-45028c5c87dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:58.551925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7c52a-a509-45d1-ba26-45028c5c87dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:58.552213 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7c52a-a509-45d1-ba26-45028c5c87dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:58.552489 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7c52a-a509-45d1-ba26-45028c5c87dd admin admin] PATH_INFO: `/v3/users/7f3f3412f42142ef9562199733dbfc6f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:58.553714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7c52a-a509-45d1-ba26-45028c5c87dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f3f3412f42142ef9562199733dbfc6f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:58.555637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7c52a-a509-45d1-ba26-45028c5c87dd admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:58.568457 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-2b1bc116-a2a7-4afc-b6b1-e970d5cb7b18 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:58.569708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2b1bc116-a2a7-4afc-b6b1-e970d5cb7b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:58.577736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1263/2484] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:58.582099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1bc116-a2a7-4afc-b6b1-e970d5cb7b18 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:58.591860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1bc116-a2a7-4afc-b6b1-e970d5cb7b18 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:58.596492 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ac3c6-f55a-48a7-9f56-6ffd89778e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:58.597124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ac3c6-f55a-48a7-9f56-6ffd89778e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:58.597435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ac3c6-f55a-48a7-9f56-6ffd89778e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:58.597708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ac3c6-f55a-48a7-9f56-6ffd89778e8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:58.598424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ac3c6-f55a-48a7-9f56-6ffd89778e8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:36:58.598732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ac3c6-f55a-48a7-9f56-6ffd89778e8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:58.600322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ac3c6-f55a-48a7-9f56-6ffd89778e8a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:58.604446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ced7c52a-a509-45d1-ba26-45028c5c87dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f3f3412f42142ef9562199733dbfc6f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:58.613486 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1264/2485] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:36:58.617674 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ac3c6-f55a-48a7-9f56-6ffd89778e8a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:58.618111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ced7c52a-a509-45d1-ba26-45028c5c87dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f3f3412f42142ef9562199733dbfc6f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:58.618815 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ced7c52a-a509-45d1-ba26-45028c5c87dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f3f3412f42142ef9562199733dbfc6f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:58.619409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ced7c52a-a509-45d1-ba26-45028c5c87dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f3f3412f42142ef9562199733dbfc6f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:58.627001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ced7c52a-a509-45d1-ba26-45028c5c87dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f3f3412f42142ef9562199733dbfc6f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:58.627697 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ac3c6-f55a-48a7-9f56-6ffd89778e8a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:58.632459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2319c4e-9ff9-42f9-b673-68af53a31f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:58.633127 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2319c4e-9ff9-42f9-b673-68af53a31f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:58.633448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2319c4e-9ff9-42f9-b673-68af53a31f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:58.633737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2319c4e-9ff9-42f9-b673-68af53a31f7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:58.634454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2319c4e-9ff9-42f9-b673-68af53a31f7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:58.636359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2319c4e-9ff9-42f9-b673-68af53a31f7e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:58.640824 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1222/2486] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:36:58 2022] DELETE /identity/v3/users/7f3f3412f42142ef9562199733dbfc6f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:58.650647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dacb4c-b96a-4486-ade4-85e3ebf140d5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:58.651017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dacb4c-b96a-4486-ade4-85e3ebf140d5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:58.651358 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dacb4c-b96a-4486-ade4-85e3ebf140d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:58.669640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1265/2487] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:36:58 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:36:58.670720 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2319c4e-9ff9-42f9-b673-68af53a31f7e admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:58.671311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c5dacb4c-b96a-4486-ade4-85e3ebf140d5 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:58.672485 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c2319c4e-9ff9-42f9-b673-68af53a31f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:58.672602 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c5dacb4c-b96a-4486-ade4-85e3ebf140d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:58.707099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:58.733737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1223/2488] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:36:58.740952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2319c4e-9ff9-42f9-b673-68af53a31f7e admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:58.743302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dacb4c-b96a-4486-ade4-85e3ebf140d5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:58.744627 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c5dacb4c-b96a-4486-ade4-85e3ebf140d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:58.746226 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4494f-7ed8-466c-85b0-83afa28554e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:58.746992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4494f-7ed8-466c-85b0-83afa28554e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:58.747275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4494f-7ed8-466c-85b0-83afa28554e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:58.747552 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4494f-7ed8-466c-85b0-83afa28554e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:58.748241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4494f-7ed8-466c-85b0-83afa28554e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:58.750195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4494f-7ed8-466c-85b0-83afa28554e8 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:58.770655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1266/2489] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:36:58 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:36:58.774656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4494f-7ed8-466c-85b0-83afa28554e8 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:58.784836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4494f-7ed8-466c-85b0-83afa28554e8 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:58.789438 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8e719-09b9-4c56-bb08-a31e454e53ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:58.790090 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8e719-09b9-4c56-bb08-a31e454e53ce admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:58.790369 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8e719-09b9-4c56-bb08-a31e454e53ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:58.790885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8e719-09b9-4c56-bb08-a31e454e53ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:58.791356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8e719-09b9-4c56-bb08-a31e454e53ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:58.793742 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8e719-09b9-4c56-bb08-a31e454e53ce admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:58.803550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1267/2490] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:36:58.807344 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8e719-09b9-4c56-bb08-a31e454e53ce admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:58.809200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dacb4c-b96a-4486-ade4-85e3ebf140d5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:58.812845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c702bc-9c9c-4a99-b44e-034ce6b4af04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:58.813527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c702bc-9c9c-4a99-b44e-034ce6b4af04 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:58.813807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c702bc-9c9c-4a99-b44e-034ce6b4af04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:58.814082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c702bc-9c9c-4a99-b44e-034ce6b4af04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:58.817416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8e719-09b9-4c56-bb08-a31e454e53ce admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:58.822205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4b52f-2d48-4eba-b427-d956b60d9076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:58.822833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4b52f-2d48-4eba-b427-d956b60d9076 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:58.823119 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4b52f-2d48-4eba-b427-d956b60d9076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:58.823327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c702bc-9c9c-4a99-b44e-034ce6b4af04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:58.823488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4b52f-2d48-4eba-b427-d956b60d9076 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:58.824282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4b52f-2d48-4eba-b427-d956b60d9076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:58.824780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c702bc-9c9c-4a99-b44e-034ce6b4af04 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:58.825892 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4b52f-2d48-4eba-b427-d956b60d9076 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:58.834711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1224/2491] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:58.838606 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1268/2492] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:36:58.840177 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c702bc-9c9c-4a99-b44e-034ce6b4af04 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:58.849833 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c702bc-9c9c-4a99-b44e-034ce6b4af04 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:58.854434 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fe63d-34cd-4cb6-ac5e-20ef415ce57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:58.856133 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fe63d-34cd-4cb6-ac5e-20ef415ce57f admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:58.856420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fe63d-34cd-4cb6-ac5e-20ef415ce57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:58.856696 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fe63d-34cd-4cb6-ac5e-20ef415ce57f admin admin] PATH_INFO: `/v3/projects/57bd5f30987e46019c30fe3da92328a4/users/ac71bf428ae84956bccf7fd820fded48/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:58.860303 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:58.876727 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fe63d-34cd-4cb6-ac5e-20ef415ce57f admin admin] RBAC: Authorizing `identity:create_grant(project_id=57bd5f30987e46019c30fe3da92328a4, user_id=ac71bf428ae84956bccf7fd820fded48, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:58.880176 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fe63d-34cd-4cb6-ac5e-20ef415ce57f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:58.902207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1225/2493] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:58 2022] PUT /identity/v3/projects/57bd5f30987e46019c30fe3da92328a4/users/ac71bf428ae84956bccf7fd820fded48/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:58.906623 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1f94a-2bde-4397-af2e-508dbf25e828 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:58.907137 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1f94a-2bde-4397-af2e-508dbf25e828 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:58.907224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1f94a-2bde-4397-af2e-508dbf25e828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:58.934973 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-5cf1f94a-2bde-4397-af2e-508dbf25e828 None None] MFA Rules not processed for user `ac71bf428ae84956bccf7fd820fded48`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:58.936195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5cf1f94a-2bde-4397-af2e-508dbf25e828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:59.002514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1269/2494] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:36:59.007086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fe63d-34cd-4cb6-ac5e-20ef415ce57f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:59.075450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fe63d-34cd-4cb6-ac5e-20ef415ce57f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:59.080871 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796fb2c-79e4-4bb9-91cc-ed3bdd99dce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:59.081578 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796fb2c-79e4-4bb9-91cc-ed3bdd99dce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:59.081867 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796fb2c-79e4-4bb9-91cc-ed3bdd99dce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:59.082147 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796fb2c-79e4-4bb9-91cc-ed3bdd99dce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:59.082849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796fb2c-79e4-4bb9-91cc-ed3bdd99dce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:59.084756 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796fb2c-79e4-4bb9-91cc-ed3bdd99dce6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:59.114836 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1f94a-2bde-4397-af2e-508dbf25e828 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:59.116301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5cf1f94a-2bde-4397-af2e-508dbf25e828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:59.120135 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1226/2495] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:36:59 2022] POST /identity/v3/projects => generated 389 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:36:59.121650 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796fb2c-79e4-4bb9-91cc-ed3bdd99dce6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:59.143740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1f94a-2bde-4397-af2e-508dbf25e828 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:59.187576 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796fb2c-79e4-4bb9-91cc-ed3bdd99dce6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:59.192263 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d548f9c-aa84-497e-9d6a-8f4e4de7c065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:59.192941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d548f9c-aa84-497e-9d6a-8f4e4de7c065 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:59.193262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d548f9c-aa84-497e-9d6a-8f4e4de7c065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:59.193545 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d548f9c-aa84-497e-9d6a-8f4e4de7c065 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:59.194255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d548f9c-aa84-497e-9d6a-8f4e4de7c065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:59.196173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d548f9c-aa84-497e-9d6a-8f4e4de7c065 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:59.206034 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a87b6-0c26-4fce-87f7-e7c58749e256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:59.206873 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a87b6-0c26-4fce-87f7-e7c58749e256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:59.207245 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a87b6-0c26-4fce-87f7-e7c58749e256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:59.207634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a87b6-0c26-4fce-87f7-e7c58749e256 admin admin] PATH_INFO: `/v3/projects/27a47546a133461baa7e1bead1810410` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:59.209228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a87b6-0c26-4fce-87f7-e7c58749e256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27a47546a133461baa7e1bead1810410)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:59.211198 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a87b6-0c26-4fce-87f7-e7c58749e256 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:59.216096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1227/2496] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:36:59 2022] POST /identity/v3/users => generated 301 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:36:59.219992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d548f9c-aa84-497e-9d6a-8f4e4de7c065 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:59.229651 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d548f9c-aa84-497e-9d6a-8f4e4de7c065 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:59.234787 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f77f7a-21ce-45a8-9aaf-9efd0755ecf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:59.235439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f77f7a-21ce-45a8-9aaf-9efd0755ecf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:59.235724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f77f7a-21ce-45a8-9aaf-9efd0755ecf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:59.236007 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f77f7a-21ce-45a8-9aaf-9efd0755ecf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:59.236743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f77f7a-21ce-45a8-9aaf-9efd0755ecf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:59.238657 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f77f7a-21ce-45a8-9aaf-9efd0755ecf9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:59.250872 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1228/2497] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:36:59.253915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:59.255381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f77f7a-21ce-45a8-9aaf-9efd0755ecf9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:59.267126 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f77f7a-21ce-45a8-9aaf-9efd0755ecf9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:59.269390 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-448a87b6-0c26-4fce-87f7-e7c58749e256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27a47546a133461baa7e1bead1810410', 'request_id': 'req-448a87b6-0c26-4fce-87f7-e7c58749e256'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:59.275229 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-448a87b6-0c26-4fce-87f7-e7c58749e256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27a47546a133461baa7e1bead1810410', 'request_id': 'req-448a87b6-0c26-4fce-87f7-e7c58749e256'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:59.281589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-448a87b6-0c26-4fce-87f7-e7c58749e256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27a47546a133461baa7e1bead1810410', 'request_id': 'req-448a87b6-0c26-4fce-87f7-e7c58749e256'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:59.294897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-448a87b6-0c26-4fce-87f7-e7c58749e256 admin admin] The token cache is being invalidate because project 27a47546a133461baa7e1bead1810410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:36:59.295272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-448a87b6-0c26-4fce-87f7-e7c58749e256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:59.295996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-448a87b6-0c26-4fce-87f7-e7c58749e256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:59.297678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1270/2498] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:36:59 2022] DELETE /identity/v3/projects/27a47546a133461baa7e1bead1810410 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:59.301531 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a87b6-0c26-4fce-87f7-e7c58749e256 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:59.302926 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-448a87b6-0c26-4fce-87f7-e7c58749e256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:59.316026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:59.329823 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74172978-95b8-48cd-9557-3782bd4eb2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:59.330835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74172978-95b8-48cd-9557-3782bd4eb2f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:59.331130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74172978-95b8-48cd-9557-3782bd4eb2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:59.331407 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74172978-95b8-48cd-9557-3782bd4eb2f2 admin admin] PATH_INFO: `/v3/projects/e33c29c9f95c4b37b841b6a4b031ee26/users/8152243239854c4992315eeefde0e4a8/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:59.346766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74172978-95b8-48cd-9557-3782bd4eb2f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e33c29c9f95c4b37b841b6a4b031ee26, user_id=8152243239854c4992315eeefde0e4a8, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:59.349488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74172978-95b8-48cd-9557-3782bd4eb2f2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:59.364273 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1229/2499] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:59 2022] PUT /identity/v3/projects/e33c29c9f95c4b37b841b6a4b031ee26/users/8152243239854c4992315eeefde0e4a8/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:59.367946 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74172978-95b8-48cd-9557-3782bd4eb2f2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:59.369788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-74172978-95b8-48cd-9557-3782bd4eb2f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:59.379640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a87b6-0c26-4fce-87f7-e7c58749e256 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:59.446993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74172978-95b8-48cd-9557-3782bd4eb2f2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:59.449200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21825f-15e6-41f5-80b8-01e700fd2c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:59.451787 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21825f-15e6-41f5-80b8-01e700fd2c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:59.451787 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21825f-15e6-41f5-80b8-01e700fd2c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:59.451787 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21825f-15e6-41f5-80b8-01e700fd2c3e admin admin] PATH_INFO: `/v3/users/0e00f0f609024462a31718a1c4173eb9` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:59.456381 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21825f-15e6-41f5-80b8-01e700fd2c3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e00f0f609024462a31718a1c4173eb9)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:59.458090 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411f3b4-b5b8-484b-8097-f44730616a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:59.458499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21825f-15e6-41f5-80b8-01e700fd2c3e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:59.458774 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411f3b4-b5b8-484b-8097-f44730616a84 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:59.459065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411f3b4-b5b8-484b-8097-f44730616a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:59.459351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411f3b4-b5b8-484b-8097-f44730616a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:59.460095 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411f3b4-b5b8-484b-8097-f44730616a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:59.461753 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411f3b4-b5b8-484b-8097-f44730616a84 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:59.476509 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1230/2500] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:36:59.477407 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411f3b4-b5b8-484b-8097-f44730616a84 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:59.492932 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411f3b4-b5b8-484b-8097-f44730616a84 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:59.493782 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e19392-4a1c-4ccb-93f5-acb1d2fdec1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:59.494349 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e19392-4a1c-4ccb-93f5-acb1d2fdec1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:59.494638 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e19392-4a1c-4ccb-93f5-acb1d2fdec1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:59.494921 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e19392-4a1c-4ccb-93f5-acb1d2fdec1e admin admin] PATH_INFO: `/v3/projects/e33c29c9f95c4b37b841b6a4b031ee26/users/8152243239854c4992315eeefde0e4a8/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:59.498105 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e19392-4a1c-4ccb-93f5-acb1d2fdec1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e33c29c9f95c4b37b841b6a4b031ee26, user_id=8152243239854c4992315eeefde0e4a8, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:59.500904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e19392-4a1c-4ccb-93f5-acb1d2fdec1e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:59.514406 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1231/2501] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:36:59 2022] PUT /identity/v3/projects/e33c29c9f95c4b37b841b6a4b031ee26/users/8152243239854c4992315eeefde0e4a8/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:36:59.522468 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-4e21825f-15e6-41f5-80b8-01e700fd2c3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e00f0f609024462a31718a1c4173eb9'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:59.526088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc061609-bf3b-4f88-b9cf-b148bb0002cb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:59.526088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc061609-bf3b-4f88-b9cf-b148bb0002cb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:59.526088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc061609-bf3b-4f88-b9cf-b148bb0002cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:59.540306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-4e21825f-15e6-41f5-80b8-01e700fd2c3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e00f0f609024462a31718a1c4173eb9'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:59.541035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-4e21825f-15e6-41f5-80b8-01e700fd2c3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e00f0f609024462a31718a1c4173eb9'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:59.544909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:36:59.553413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-4e21825f-15e6-41f5-80b8-01e700fd2c3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e00f0f609024462a31718a1c4173eb9'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:59.554308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-4e21825f-15e6-41f5-80b8-01e700fd2c3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e00f0f609024462a31718a1c4173eb9'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:36:59.554760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-dc061609-bf3b-4f88-b9cf-b148bb0002cb None None] MFA Rules not processed for user `8152243239854c4992315eeefde0e4a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:36:59.556071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-dc061609-bf3b-4f88-b9cf-b148bb0002cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:59.569824 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1271/2502] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:36:59 2022] DELETE /identity/v3/users/0e00f0f609024462a31718a1c4173eb9 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:36:59.650468 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1232/2503] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:36:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:36:59.663136 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21825f-15e6-41f5-80b8-01e700fd2c3e admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:59.664802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-4e21825f-15e6-41f5-80b8-01e700fd2c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:59.738840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21825f-15e6-41f5-80b8-01e700fd2c3e admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:36:59.742843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe58b0-52e5-412a-8b0b-d20f5611ccb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:36:59.743616 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe58b0-52e5-412a-8b0b-d20f5611ccb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:36:59.743966 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe58b0-52e5-412a-8b0b-d20f5611ccb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:36:59.744312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe58b0-52e5-412a-8b0b-d20f5611ccb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:36:59.825350 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe58b0-52e5-412a-8b0b-d20f5611ccb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:36:59.826812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe58b0-52e5-412a-8b0b-d20f5611ccb4 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:36:59.841398 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1272/2504] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:36:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:36:59.925615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc061609-bf3b-4f88-b9cf-b148bb0002cb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:59.927113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-dc061609-bf3b-4f88-b9cf-b148bb0002cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:59.977738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe58b0-52e5-412a-8b0b-d20f5611ccb4 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:36:59.979195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1fbe58b0-52e5-412a-8b0b-d20f5611ccb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:36:59.984618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:00.008658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc061609-bf3b-4f88-b9cf-b148bb0002cb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:00.049122 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe58b0-52e5-412a-8b0b-d20f5611ccb4 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:00.053855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832a2f7-1f85-479b-813d-3501c3734831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:00.054885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832a2f7-1f85-479b-813d-3501c3734831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.055232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832a2f7-1f85-479b-813d-3501c3734831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.055513 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832a2f7-1f85-479b-813d-3501c3734831 admin admin] PATH_INFO: `/v3/users/ac71bf428ae84956bccf7fd820fded48` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.056766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832a2f7-1f85-479b-813d-3501c3734831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac71bf428ae84956bccf7fd820fded48)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:00.059713 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832a2f7-1f85-479b-813d-3501c3734831 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:00.066285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ad9cb-43d3-4741-827f-9b6a20e7dffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:00.067148 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ad9cb-43d3-4741-827f-9b6a20e7dffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.067435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ad9cb-43d3-4741-827f-9b6a20e7dffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.067744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ad9cb-43d3-4741-827f-9b6a20e7dffc admin admin] PATH_INFO: `/v3/projects/73d34046acab4b98a834a66ae712a8f3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.068927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ad9cb-43d3-4741-827f-9b6a20e7dffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=73d34046acab4b98a834a66ae712a8f3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:00.070915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ad9cb-43d3-4741-827f-9b6a20e7dffc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:00.100860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:DELETE=6,keystone:SELECT=94 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:00.111088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a832a2f7-1f85-479b-813d-3501c3734831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac71bf428ae84956bccf7fd820fded48'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:00.123732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-848ad9cb-43d3-4741-827f-9b6a20e7dffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73d34046acab4b98a834a66ae712a8f3', 'request_id': 'req-848ad9cb-43d3-4741-827f-9b6a20e7dffc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:00.124019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a832a2f7-1f85-479b-813d-3501c3734831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac71bf428ae84956bccf7fd820fded48'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:00.124689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a832a2f7-1f85-479b-813d-3501c3734831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac71bf428ae84956bccf7fd820fded48'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:00.131811 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-848ad9cb-43d3-4741-827f-9b6a20e7dffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73d34046acab4b98a834a66ae712a8f3', 'request_id': 'req-848ad9cb-43d3-4741-827f-9b6a20e7dffc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:00.134416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a832a2f7-1f85-479b-813d-3501c3734831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac71bf428ae84956bccf7fd820fded48'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:00.135050 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a832a2f7-1f85-479b-813d-3501c3734831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac71bf428ae84956bccf7fd820fded48'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:00.137876 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-848ad9cb-43d3-4741-827f-9b6a20e7dffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73d34046acab4b98a834a66ae712a8f3', 'request_id': 'req-848ad9cb-43d3-4741-827f-9b6a20e7dffc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:00.147845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1273/2505] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:36:59 2022] DELETE /identity/v3/users/ac71bf428ae84956bccf7fd820fded48 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:00.152089 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-848ad9cb-43d3-4741-827f-9b6a20e7dffc admin admin] The token cache is being invalidate because project 73d34046acab4b98a834a66ae712a8f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:00.152299 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081d0f3-791d-4237-a533-ddb76c581bcd None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.152435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-848ad9cb-43d3-4741-827f-9b6a20e7dffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:00.152520 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081d0f3-791d-4237-a533-ddb76c581bcd None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.152720 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081d0f3-791d-4237-a533-ddb76c581bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.153048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-848ad9cb-43d3-4741-827f-9b6a20e7dffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:00.154777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1233/2506] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:36:59 2022] DELETE /identity/v3/projects/73d34046acab4b98a834a66ae712a8f3 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:00.159082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1412eb-fa1c-4805-83a6-5b23de2a32a8 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.159367 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1412eb-fa1c-4805-83a6-5b23de2a32a8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.159646 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1412eb-fa1c-4805-83a6-5b23de2a32a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.175021 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-9081d0f3-791d-4237-a533-ddb76c581bcd None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:00.176278 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9081d0f3-791d-4237-a533-ddb76c581bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:00.178299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6c1412eb-fa1c-4805-83a6-5b23de2a32a8 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:00.179483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6c1412eb-fa1c-4805-83a6-5b23de2a32a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:00.236965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1274/2507] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:00.244607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1234/2508] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:00.247591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081d0f3-791d-4237-a533-ddb76c581bcd None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:00.252027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9081d0f3-791d-4237-a533-ddb76c581bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:00.252753 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade58c74-e47f-4da9-a7bf-46c65dc54ad3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.253046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade58c74-e47f-4da9-a7bf-46c65dc54ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.253393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade58c74-e47f-4da9-a7bf-46c65dc54ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.272772 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ade58c74-e47f-4da9-a7bf-46c65dc54ad3 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:00.274029 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ade58c74-e47f-4da9-a7bf-46c65dc54ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:00.282821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1235/2509] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:00.286858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade58c74-e47f-4da9-a7bf-46c65dc54ad3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:00.297129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade58c74-e47f-4da9-a7bf-46c65dc54ad3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:00.301889 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d417a-9a99-4374-83a6-2c5aa348f909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:00.302539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d417a-9a99-4374-83a6-2c5aa348f909 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.302887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d417a-9a99-4374-83a6-2c5aa348f909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.303127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d417a-9a99-4374-83a6-2c5aa348f909 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.303859 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d417a-9a99-4374-83a6-2c5aa348f909 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:37:00.304169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d417a-9a99-4374-83a6-2c5aa348f909 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:00.305848 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d417a-9a99-4374-83a6-2c5aa348f909 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:00.311269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081d0f3-791d-4237-a533-ddb76c581bcd None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:00.312800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:00.315611 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92be340-9d13-4e70-b219-f4f6d8125cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:00.316259 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92be340-9d13-4e70-b219-f4f6d8125cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.316542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92be340-9d13-4e70-b219-f4f6d8125cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.316816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92be340-9d13-4e70-b219-f4f6d8125cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.319588 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1236/2510] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:37:00.323771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d417a-9a99-4374-83a6-2c5aa348f909 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:00.329631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92be340-9d13-4e70-b219-f4f6d8125cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:00.331104 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92be340-9d13-4e70-b219-f4f6d8125cf8 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:00.333609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d417a-9a99-4374-83a6-2c5aa348f909 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:00.338157 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955f05b-0ff5-4bc7-ab2f-5db4701dd7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:00.338783 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955f05b-0ff5-4bc7-ab2f-5db4701dd7f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.339662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955f05b-0ff5-4bc7-ab2f-5db4701dd7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.340036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955f05b-0ff5-4bc7-ab2f-5db4701dd7f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.340739 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955f05b-0ff5-4bc7-ab2f-5db4701dd7f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:00.341442 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1275/2511] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:00.346495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955f05b-0ff5-4bc7-ab2f-5db4701dd7f6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:00.376454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1237/2512] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:00 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:37:00.377747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92be340-9d13-4e70-b219-f4f6d8125cf8 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:00.388307 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92be340-9d13-4e70-b219-f4f6d8125cf8 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:00.449459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:00.457678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b28116-a3b4-4483-b159-32eda72e81a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:00.459139 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28116-a3b4-4483-b159-32eda72e81a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.459598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28116-a3b4-4483-b159-32eda72e81a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.460023 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28116-a3b4-4483-b159-32eda72e81a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.460783 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b28116-a3b4-4483-b159-32eda72e81a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:00.463150 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b28116-a3b4-4483-b159-32eda72e81a7 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:00.483948 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1276/2513] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:00 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:00.487940 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955f05b-0ff5-4bc7-ab2f-5db4701dd7f6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:00.500458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955f05b-0ff5-4bc7-ab2f-5db4701dd7f6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:00.505268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1be57-78de-4481-8f2e-73787e20ca2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:00.505945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1be57-78de-4481-8f2e-73787e20ca2a admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.506603 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1be57-78de-4481-8f2e-73787e20ca2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.506603 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1be57-78de-4481-8f2e-73787e20ca2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.507211 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1be57-78de-4481-8f2e-73787e20ca2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:00.508802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1be57-78de-4481-8f2e-73787e20ca2a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:00.520023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1238/2514] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:37:00.524064 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b28116-a3b4-4483-b159-32eda72e81a7 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:00.534904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b28116-a3b4-4483-b159-32eda72e81a7 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:00.539598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3491194-a499-485a-bc7f-044e51ee1282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:00.540266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3491194-a499-485a-bc7f-044e51ee1282 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.540555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3491194-a499-485a-bc7f-044e51ee1282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.540866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3491194-a499-485a-bc7f-044e51ee1282 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.541683 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3491194-a499-485a-bc7f-044e51ee1282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:00.544207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3491194-a499-485a-bc7f-044e51ee1282 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:00.557956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1277/2515] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:00.559561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1be57-78de-4481-8f2e-73787e20ca2a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:00.569601 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1be57-78de-4481-8f2e-73787e20ca2a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:00.574775 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcdab7-0ff5-43f9-8c4f-92466157d9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:00.575814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcdab7-0ff5-43f9-8c4f-92466157d9c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.576127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcdab7-0ff5-43f9-8c4f-92466157d9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.576441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcdab7-0ff5-43f9-8c4f-92466157d9c1 admin admin] PATH_INFO: `/v3/projects/5ee7b79658734ca2b0eaecb3c0d470c6/users/74ae420841b9421998a630f59c4cb5a0/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.593238 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbcdab7-0ff5-43f9-8c4f-92466157d9c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ee7b79658734ca2b0eaecb3c0d470c6, user_id=74ae420841b9421998a630f59c4cb5a0, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:00.596547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbcdab7-0ff5-43f9-8c4f-92466157d9c1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:00.611169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1239/2516] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:00 2022] PUT /identity/v3/projects/5ee7b79658734ca2b0eaecb3c0d470c6/users/74ae420841b9421998a630f59c4cb5a0/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:00.623648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a94b6-bbf8-41d9-adbb-adb4b46baa61 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.623648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a94b6-bbf8-41d9-adbb-adb4b46baa61 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.623979 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a94b6-bbf8-41d9-adbb-adb4b46baa61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.632802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcdab7-0ff5-43f9-8c4f-92466157d9c1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:00.634592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-adbcdab7-0ff5-43f9-8c4f-92466157d9c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:00.663544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-5b6a94b6-bbf8-41d9-adbb-adb4b46baa61 None None] MFA Rules not processed for user `74ae420841b9421998a630f59c4cb5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:00.664868 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5b6a94b6-bbf8-41d9-adbb-adb4b46baa61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:00.708890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcdab7-0ff5-43f9-8c4f-92466157d9c1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:00.713777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:00.715206 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.715492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.715765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4 admin admin] PATH_INFO: `/v3/projects/57bd5f30987e46019c30fe3da92328a4` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.716985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57bd5f30987e46019c30fe3da92328a4)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:00.718987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:00.736133 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1278/2517] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:00.740596 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a94b6-bbf8-41d9-adbb-adb4b46baa61 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:00.752142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a94b6-bbf8-41d9-adbb-adb4b46baa61 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:00.766271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57bd5f30987e46019c30fe3da92328a4', 'request_id': 'req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:00.774488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57bd5f30987e46019c30fe3da92328a4', 'request_id': 'req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:00.782283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57bd5f30987e46019c30fe3da92328a4', 'request_id': 'req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:00.793950 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4 admin admin] The token cache is being invalidate because project 57bd5f30987e46019c30fe3da92328a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:00.794246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:00.794964 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:00.796530 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1240/2518] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:00 2022] DELETE /identity/v3/projects/57bd5f30987e46019c30fe3da92328a4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:00.800465 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:00.802247 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:00.810701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc23a06-13f4-4a7a-9094-92017d8c7785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:00.811494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc23a06-13f4-4a7a-9094-92017d8c7785 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.811784 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc23a06-13f4-4a7a-9094-92017d8c7785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.812059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc23a06-13f4-4a7a-9094-92017d8c7785 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.814055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc23a06-13f4-4a7a-9094-92017d8c7785 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:00.814709 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc23a06-13f4-4a7a-9094-92017d8c7785 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:00.849859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1279/2519] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:00 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:37:00.850806 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc23a06-13f4-4a7a-9094-92017d8c7785 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:00.852502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-6bc23a06-13f4-4a7a-9094-92017d8c7785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:00.854005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:00.899467 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb526f62-f1fa-4768-91ae-d0bd0c4481c4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:00.927272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:00.943181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc23a06-13f4-4a7a-9094-92017d8c7785 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:00.948147 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8b0ec-fe87-4d7c-8c94-8f57b5061851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:00.949695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8b0ec-fe87-4d7c-8c94-8f57b5061851 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.950055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8b0ec-fe87-4d7c-8c94-8f57b5061851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.950405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8b0ec-fe87-4d7c-8c94-8f57b5061851 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.951173 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8b0ec-fe87-4d7c-8c94-8f57b5061851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:00.953202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8b0ec-fe87-4d7c-8c94-8f57b5061851 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:00.966037 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d9999-05b3-47ca-a17e-007297e65eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:00.966914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d9999-05b3-47ca-a17e-007297e65eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:00.967200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d9999-05b3-47ca-a17e-007297e65eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:00.967483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d9999-05b3-47ca-a17e-007297e65eec admin admin] PATH_INFO: `/v3/users/8152243239854c4992315eeefde0e4a8` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:00.968698 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d9999-05b3-47ca-a17e-007297e65eec admin admin] RBAC: Authorizing `identity:delete_user(user_id=8152243239854c4992315eeefde0e4a8)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:00.970685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d9999-05b3-47ca-a17e-007297e65eec admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:00.974927 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1280/2520] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:00 2022] POST /identity/v3/users => generated 300 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:00.975737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8b0ec-fe87-4d7c-8c94-8f57b5061851 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:00.977335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c2a8b0ec-fe87-4d7c-8c94-8f57b5061851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:01.022795 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-690d9999-05b3-47ca-a17e-007297e65eec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8152243239854c4992315eeefde0e4a8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:01.035676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-690d9999-05b3-47ca-a17e-007297e65eec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8152243239854c4992315eeefde0e4a8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:01.036493 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-690d9999-05b3-47ca-a17e-007297e65eec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8152243239854c4992315eeefde0e4a8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:01.036994 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8b0ec-fe87-4d7c-8c94-8f57b5061851 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:01.037559 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-690d9999-05b3-47ca-a17e-007297e65eec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8152243239854c4992315eeefde0e4a8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:01.040695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a8881-ac64-4df2-bd7f-ee7ffa852e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:01.041376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a8881-ac64-4df2-bd7f-ee7ffa852e54 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:01.041657 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a8881-ac64-4df2-bd7f-ee7ffa852e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:01.041934 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a8881-ac64-4df2-bd7f-ee7ffa852e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:01.043903 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b73a8881-ac64-4df2-bd7f-ee7ffa852e54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:01.044247 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-690d9999-05b3-47ca-a17e-007297e65eec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8152243239854c4992315eeefde0e4a8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:01.058926 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a8881-ac64-4df2-bd7f-ee7ffa852e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:01.060395 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a8881-ac64-4df2-bd7f-ee7ffa852e54 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:01.060544 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1241/2521] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:00 2022] DELETE /identity/v3/users/8152243239854c4992315eeefde0e4a8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:01.063746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d9999-05b3-47ca-a17e-007297e65eec admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:01.063746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-690d9999-05b3-47ca-a17e-007297e65eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:01.072612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1281/2522] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:01.133649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:01.138252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d9999-05b3-47ca-a17e-007297e65eec admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:01.144956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929ff61-bb9c-411c-bd68-dca33f5231ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:01.147005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929ff61-bb9c-411c-bd68-dca33f5231ef admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:01.147005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929ff61-bb9c-411c-bd68-dca33f5231ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:01.147005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929ff61-bb9c-411c-bd68-dca33f5231ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:01.147569 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929ff61-bb9c-411c-bd68-dca33f5231ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:01.149225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929ff61-bb9c-411c-bd68-dca33f5231ef admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:01.160504 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1242/2523] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:01.165502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a8881-ac64-4df2-bd7f-ee7ffa852e54 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:01.175846 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a8881-ac64-4df2-bd7f-ee7ffa852e54 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:01.182173 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007238c-015f-47a9-a6c4-afba625df6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:01.183954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007238c-015f-47a9-a6c4-afba625df6bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:01.184337 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007238c-015f-47a9-a6c4-afba625df6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:01.184690 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007238c-015f-47a9-a6c4-afba625df6bc admin admin] PATH_INFO: `/v3/projects/68b9e048392d467f8ef20bd2c8fc85c7/users/9b6fa3e948534d7e9f03ee50fea0732f/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:01.201450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007238c-015f-47a9-a6c4-afba625df6bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=68b9e048392d467f8ef20bd2c8fc85c7, user_id=9b6fa3e948534d7e9f03ee50fea0732f, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:01.204433 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007238c-015f-47a9-a6c4-afba625df6bc admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:01.218913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1282/2524] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:01 2022] PUT /identity/v3/projects/68b9e048392d467f8ef20bd2c8fc85c7/users/9b6fa3e948534d7e9f03ee50fea0732f/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:01.223176 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929ff61-bb9c-411c-bd68-dca33f5231ef admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:01.234122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929ff61-bb9c-411c-bd68-dca33f5231ef admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:01.302915 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef47f6-19fb-4ee5-b0f1-26a85293f301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:01.303504 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef47f6-19fb-4ee5-b0f1-26a85293f301 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:01.303786 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef47f6-19fb-4ee5-b0f1-26a85293f301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:01.304061 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef47f6-19fb-4ee5-b0f1-26a85293f301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:01.304788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef47f6-19fb-4ee5-b0f1-26a85293f301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:01.306489 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef47f6-19fb-4ee5-b0f1-26a85293f301 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:01.320917 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1243/2525] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:01.321537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007238c-015f-47a9-a6c4-afba625df6bc admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:01.334513 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007238c-015f-47a9-a6c4-afba625df6bc admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:01.339756 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443a2ab-b590-4c31-8864-89821962cae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:01.340724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443a2ab-b590-4c31-8864-89821962cae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:01.341013 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443a2ab-b590-4c31-8864-89821962cae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:01.341432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443a2ab-b590-4c31-8864-89821962cae0 admin admin] PATH_INFO: `/v3/projects/68b9e048392d467f8ef20bd2c8fc85c7/users/9b6fa3e948534d7e9f03ee50fea0732f/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:01.343590 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443a2ab-b590-4c31-8864-89821962cae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68b9e048392d467f8ef20bd2c8fc85c7, user_id=9b6fa3e948534d7e9f03ee50fea0732f, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:01.346469 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443a2ab-b590-4c31-8864-89821962cae0 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:01.357376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1283/2526] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:01 2022] PUT /identity/v3/projects/68b9e048392d467f8ef20bd2c8fc85c7/users/9b6fa3e948534d7e9f03ee50fea0732f/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:01.368417 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030a1b4-6394-489f-845c-8f6d273a0a18 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:01.368713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030a1b4-6394-489f-845c-8f6d273a0a18 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:01.368999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030a1b4-6394-489f-845c-8f6d273a0a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:01.398351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1030a1b4-6394-489f-845c-8f6d273a0a18 None None] MFA Rules not processed for user `9b6fa3e948534d7e9f03ee50fea0732f`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:01.399616 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1030a1b4-6394-489f-845c-8f6d273a0a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:01.493772 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1244/2527] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:01.505970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443a2ab-b590-4c31-8864-89821962cae0 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:01.508145 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-8443a2ab-b590-4c31-8864-89821962cae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:01.549752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030a1b4-6394-489f-845c-8f6d273a0a18 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:01.552540 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1030a1b4-6394-489f-845c-8f6d273a0a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:01.586513 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443a2ab-b590-4c31-8864-89821962cae0 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:01.590678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4163e2-34d1-4f7e-95b7-3c8d2c83d6b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:01.591394 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4163e2-34d1-4f7e-95b7-3c8d2c83d6b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:01.591706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4163e2-34d1-4f7e-95b7-3c8d2c83d6b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:01.592031 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4163e2-34d1-4f7e-95b7-3c8d2c83d6b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:01.603019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4163e2-34d1-4f7e-95b7-3c8d2c83d6b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:01.605074 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4163e2-34d1-4f7e-95b7-3c8d2c83d6b7 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:01.615727 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1284/2528] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:01.638635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030a1b4-6394-489f-845c-8f6d273a0a18 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:01.647185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d18401-06b7-42f8-8919-7d80732f705c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:01.647185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d18401-06b7-42f8-8919-7d80732f705c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:01.647185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d18401-06b7-42f8-8919-7d80732f705c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:01.647185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d18401-06b7-42f8-8919-7d80732f705c admin admin] PATH_INFO: `/v3/projects/e33c29c9f95c4b37b841b6a4b031ee26` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:01.647577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d18401-06b7-42f8-8919-7d80732f705c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e33c29c9f95c4b37b841b6a4b031ee26)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:01.648897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4163e2-34d1-4f7e-95b7-3c8d2c83d6b7 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:01.649521 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d18401-06b7-42f8-8919-7d80732f705c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:01.659902 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4163e2-34d1-4f7e-95b7-3c8d2c83d6b7 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:01.665122 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71c84a-e489-4cee-b6ee-a8dea8741d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:01.665785 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71c84a-e489-4cee-b6ee-a8dea8741d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:01.666067 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71c84a-e489-4cee-b6ee-a8dea8741d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:01.666341 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71c84a-e489-4cee-b6ee-a8dea8741d58 admin admin] PATH_INFO: `/v3/users/74ae420841b9421998a630f59c4cb5a0` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:01.668435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71c84a-e489-4cee-b6ee-a8dea8741d58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74ae420841b9421998a630f59c4cb5a0)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:01.670571 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71c84a-e489-4cee-b6ee-a8dea8741d58 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:01.672382 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:01.713324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-39d18401-06b7-42f8-8919-7d80732f705c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e33c29c9f95c4b37b841b6a4b031ee26', 'request_id': 'req-39d18401-06b7-42f8-8919-7d80732f705c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:01.722850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:01.724151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-39d18401-06b7-42f8-8919-7d80732f705c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e33c29c9f95c4b37b841b6a4b031ee26', 'request_id': 'req-39d18401-06b7-42f8-8919-7d80732f705c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:01.726332 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7b71c84a-e489-4cee-b6ee-a8dea8741d58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74ae420841b9421998a630f59c4cb5a0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:01.731009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-39d18401-06b7-42f8-8919-7d80732f705c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e33c29c9f95c4b37b841b6a4b031ee26', 'request_id': 'req-39d18401-06b7-42f8-8919-7d80732f705c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:01.742998 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-39d18401-06b7-42f8-8919-7d80732f705c admin admin] The token cache is being invalidate because project e33c29c9f95c4b37b841b6a4b031ee26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:01.745026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-39d18401-06b7-42f8-8919-7d80732f705c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:01.745026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-39d18401-06b7-42f8-8919-7d80732f705c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:01.746342 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1245/2529] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:01 2022] DELETE /identity/v3/projects/e33c29c9f95c4b37b841b6a4b031ee26 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:01.751788 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7b71c84a-e489-4cee-b6ee-a8dea8741d58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74ae420841b9421998a630f59c4cb5a0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:01.752169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ef25e-683a-486d-8168-b6f0d40597dd None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:01.752580 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ef25e-683a-486d-8168-b6f0d40597dd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:01.752689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7b71c84a-e489-4cee-b6ee-a8dea8741d58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74ae420841b9421998a630f59c4cb5a0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:01.752859 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ef25e-683a-486d-8168-b6f0d40597dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:01.763350 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7b71c84a-e489-4cee-b6ee-a8dea8741d58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74ae420841b9421998a630f59c4cb5a0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:01.764306 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7b71c84a-e489-4cee-b6ee-a8dea8741d58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74ae420841b9421998a630f59c4cb5a0'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:01.781830 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4a8ef25e-683a-486d-8168-b6f0d40597dd None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:01.783253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4a8ef25e-683a-486d-8168-b6f0d40597dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:01.789472 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1285/2530] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:01 2022] DELETE /identity/v3/users/74ae420841b9421998a630f59c4cb5a0 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:01.793655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350d866-5d5a-40fd-9768-15ad2cae6855 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:01.795068 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350d866-5d5a-40fd-9768-15ad2cae6855 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:01.795396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350d866-5d5a-40fd-9768-15ad2cae6855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:01.815866 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-a350d866-5d5a-40fd-9768-15ad2cae6855 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:01.817604 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a350d866-5d5a-40fd-9768-15ad2cae6855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:01.876804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1246/2531] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:01.887820 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25435217-dd49-43f4-bbdb-5192f47a9500 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:01.888125 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25435217-dd49-43f4-bbdb-5192f47a9500 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:01.888618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25435217-dd49-43f4-bbdb-5192f47a9500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:01.914327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-25435217-dd49-43f4-bbdb-5192f47a9500 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:01.915408 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1286/2532] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:01.921015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-25435217-dd49-43f4-bbdb-5192f47a9500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:01.926104 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350d866-5d5a-40fd-9768-15ad2cae6855 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:01.927558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a350d866-5d5a-40fd-9768-15ad2cae6855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:01.930987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1247/2533] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:01.934989 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25435217-dd49-43f4-bbdb-5192f47a9500 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:01.944822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25435217-dd49-43f4-bbdb-5192f47a9500 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:01.949951 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3049ea6-f49c-4524-8546-4ff5efccfd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:01.950619 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3049ea6-f49c-4524-8546-4ff5efccfd84 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:01.950908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3049ea6-f49c-4524-8546-4ff5efccfd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:01.951327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3049ea6-f49c-4524-8546-4ff5efccfd84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:01.952290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3049ea6-f49c-4524-8546-4ff5efccfd84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:37:01.952611 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3049ea6-f49c-4524-8546-4ff5efccfd84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:01.954312 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3049ea6-f49c-4524-8546-4ff5efccfd84 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:01.969909 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1248/2534] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:37:01.974872 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3049ea6-f49c-4524-8546-4ff5efccfd84 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:01.987659 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3049ea6-f49c-4524-8546-4ff5efccfd84 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:01.998030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d71dc-3a85-4b96-8cf7-1a3509d40b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:01.999265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d71dc-3a85-4b96-8cf7-1a3509d40b68 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:02.001495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d71dc-3a85-4b96-8cf7-1a3509d40b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:02.002254 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d71dc-3a85-4b96-8cf7-1a3509d40b68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:02.004597 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d71dc-3a85-4b96-8cf7-1a3509d40b68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:02.007230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d71dc-3a85-4b96-8cf7-1a3509d40b68 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:02.007697 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350d866-5d5a-40fd-9768-15ad2cae6855 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:02.012103 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1e506-c905-4a47-83cd-d00a6920045a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:02.012868 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1e506-c905-4a47-83cd-d00a6920045a service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:02.013193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1e506-c905-4a47-83cd-d00a6920045a service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:02.013484 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1e506-c905-4a47-83cd-d00a6920045a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:02.027579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1e506-c905-4a47-83cd-d00a6920045a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:02.028763 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1e506-c905-4a47-83cd-d00a6920045a service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:02.039649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1287/2535] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:02.049190 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1249/2536] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:01 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:02.052724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1e506-c905-4a47-83cd-d00a6920045a service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:02.063996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1e506-c905-4a47-83cd-d00a6920045a service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:02.173638 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb9b1a-c3e7-4945-8cef-9663393f010d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:02.175547 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb9b1a-c3e7-4945-8cef-9663393f010d admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:02.176448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb9b1a-c3e7-4945-8cef-9663393f010d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:02.176802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb9b1a-c3e7-4945-8cef-9663393f010d admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:02.178116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb9b1a-c3e7-4945-8cef-9663393f010d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:02.183875 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb9b1a-c3e7-4945-8cef-9663393f010d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:02.208351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:02.242798 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1288/2537] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:02 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:37:02.247860 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d71dc-3a85-4b96-8cf7-1a3509d40b68 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:02.260705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d71dc-3a85-4b96-8cf7-1a3509d40b68 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:02.266888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cbb2d-a0dd-4968-94c4-95e5d28a4355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:02.267585 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cbb2d-a0dd-4968-94c4-95e5d28a4355 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:02.267873 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cbb2d-a0dd-4968-94c4-95e5d28a4355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:02.268155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cbb2d-a0dd-4968-94c4-95e5d28a4355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:02.269223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cbb2d-a0dd-4968-94c4-95e5d28a4355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:02.271048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cbb2d-a0dd-4968-94c4-95e5d28a4355 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:02.284079 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1250/2538] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:02.288272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb9b1a-c3e7-4945-8cef-9663393f010d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:02.303029 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb9b1a-c3e7-4945-8cef-9663393f010d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:02.309093 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae926a1-eb2d-40c9-9bda-ee55330fd272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:02.309880 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae926a1-eb2d-40c9-9bda-ee55330fd272 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:02.310252 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae926a1-eb2d-40c9-9bda-ee55330fd272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:02.310618 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae926a1-eb2d-40c9-9bda-ee55330fd272 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:02.311456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae926a1-eb2d-40c9-9bda-ee55330fd272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:02.313675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae926a1-eb2d-40c9-9bda-ee55330fd272 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:02.326792 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1289/2539] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:37:02.330524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cbb2d-a0dd-4968-94c4-95e5d28a4355 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:02.347744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cbb2d-a0dd-4968-94c4-95e5d28a4355 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:02.354336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080a10a-7d3f-45e5-9884-3fc009e481e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:02.356771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080a10a-7d3f-45e5-9884-3fc009e481e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:02.357058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080a10a-7d3f-45e5-9884-3fc009e481e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:02.357861 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080a10a-7d3f-45e5-9884-3fc009e481e0 admin admin] PATH_INFO: `/v3/projects/5bb5042a3bb24a8faf2f9e131cfbfec4/users/94b33546c2fd4212a2f05566862876ea/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:02.380350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080a10a-7d3f-45e5-9884-3fc009e481e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bb5042a3bb24a8faf2f9e131cfbfec4, user_id=94b33546c2fd4212a2f05566862876ea, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:02.385378 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080a10a-7d3f-45e5-9884-3fc009e481e0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:02.412565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1251/2540] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:02 2022] PUT /identity/v3/projects/5bb5042a3bb24a8faf2f9e131cfbfec4/users/94b33546c2fd4212a2f05566862876ea/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:02.414201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae926a1-eb2d-40c9-9bda-ee55330fd272 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:02.416742 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1ae926a1-eb2d-40c9-9bda-ee55330fd272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:02.426922 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7135b72-faa1-44d6-9f99-b1c21273491e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:02.427324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7135b72-faa1-44d6-9f99-b1c21273491e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:02.428435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7135b72-faa1-44d6-9f99-b1c21273491e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:02.436920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae926a1-eb2d-40c9-9bda-ee55330fd272 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:02.473557 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a7135b72-faa1-44d6-9f99-b1c21273491e None None] MFA Rules not processed for user `94b33546c2fd4212a2f05566862876ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:02.476387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a7135b72-faa1-44d6-9f99-b1c21273491e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:02.506678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:02.528680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:02.529913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:02.530198 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:02.530577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9 admin admin] PATH_INFO: `/v3/projects/5ee7b79658734ca2b0eaecb3c0d470c6` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:02.531883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ee7b79658734ca2b0eaecb3c0d470c6)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:02.533921 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:02.566544 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1252/2541] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:02.578536 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7135b72-faa1-44d6-9f99-b1c21273491e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:02.580017 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a7135b72-faa1-44d6-9f99-b1c21273491e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:02.592321 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ee7b79658734ca2b0eaecb3c0d470c6', 'request_id': 'req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:02.598176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ee7b79658734ca2b0eaecb3c0d470c6', 'request_id': 'req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:02.606987 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ee7b79658734ca2b0eaecb3c0d470c6', 'request_id': 'req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:02.623658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9 admin admin] The token cache is being invalidate because project 5ee7b79658734ca2b0eaecb3c0d470c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:02.623913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:02.624788 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:02.626616 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1290/2542] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:02 2022] DELETE /identity/v3/projects/5ee7b79658734ca2b0eaecb3c0d470c6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:02.630520 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:02.633284 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:02.647394 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7135b72-faa1-44d6-9f99-b1c21273491e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:02.705375 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6daf9b-9c2e-43bb-a8d7-655237a3a5bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:02.706194 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6daf9b-9c2e-43bb-a8d7-655237a3a5bb service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:02.706583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6daf9b-9c2e-43bb-a8d7-655237a3a5bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:02.706962 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6daf9b-9c2e-43bb-a8d7-655237a3a5bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:02.708469 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8f674-a47c-4fc3-bcd1-95e672ab85f9 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:02.709753 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ce6daf9b-9c2e-43bb-a8d7-655237a3a5bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:02.713214 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6fc33-7d5a-4a17-9d9f-29de9da0d5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:02.714904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6fc33-7d5a-4a17-9d9f-29de9da0d5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:02.715191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6fc33-7d5a-4a17-9d9f-29de9da0d5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:02.715469 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6fc33-7d5a-4a17-9d9f-29de9da0d5d8 admin admin] PATH_INFO: `/v3/users/9b6fa3e948534d7e9f03ee50fea0732f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:02.716693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6fc33-7d5a-4a17-9d9f-29de9da0d5d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b6fa3e948534d7e9f03ee50fea0732f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:02.718666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6fc33-7d5a-4a17-9d9f-29de9da0d5d8 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:02.736918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:INSERT=2,keystone:SELECT=90,keystone:DELETE=8 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:02.775655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7ab6fc33-7d5a-4a17-9d9f-29de9da0d5d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b6fa3e948534d7e9f03ee50fea0732f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:02.788085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6daf9b-9c2e-43bb-a8d7-655237a3a5bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:02.791232 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6daf9b-9c2e-43bb-a8d7-655237a3a5bb service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:02.794024 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7ab6fc33-7d5a-4a17-9d9f-29de9da0d5d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b6fa3e948534d7e9f03ee50fea0732f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:02.795047 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7ab6fc33-7d5a-4a17-9d9f-29de9da0d5d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b6fa3e948534d7e9f03ee50fea0732f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:02.803723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1253/2543] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:37:02.806730 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7ab6fc33-7d5a-4a17-9d9f-29de9da0d5d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b6fa3e948534d7e9f03ee50fea0732f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:02.807654 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7ab6fc33-7d5a-4a17-9d9f-29de9da0d5d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b6fa3e948534d7e9f03ee50fea0732f'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:02.825182 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1291/2544] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:02 2022] DELETE /identity/v3/users/9b6fa3e948534d7e9f03ee50fea0732f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:03.232339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6daf9b-9c2e-43bb-a8d7-655237a3a5bb service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:03.249474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6daf9b-9c2e-43bb-a8d7-655237a3a5bb service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:03.321280 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:03.373368 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623d65f-66a9-417c-a50f-c3d28de8b575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:03.374744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623d65f-66a9-417c-a50f-c3d28de8b575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:03.375217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623d65f-66a9-417c-a50f-c3d28de8b575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:03.375995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623d65f-66a9-417c-a50f-c3d28de8b575 admin admin] PATH_INFO: `/v3/projects/68b9e048392d467f8ef20bd2c8fc85c7` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:03.377732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623d65f-66a9-417c-a50f-c3d28de8b575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68b9e048392d467f8ef20bd2c8fc85c7)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:03.380787 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623d65f-66a9-417c-a50f-c3d28de8b575 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:03.458445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0623d65f-66a9-417c-a50f-c3d28de8b575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68b9e048392d467f8ef20bd2c8fc85c7', 'request_id': 'req-0623d65f-66a9-417c-a50f-c3d28de8b575'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:03.466562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0623d65f-66a9-417c-a50f-c3d28de8b575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68b9e048392d467f8ef20bd2c8fc85c7', 'request_id': 'req-0623d65f-66a9-417c-a50f-c3d28de8b575'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:03.473518 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0623d65f-66a9-417c-a50f-c3d28de8b575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68b9e048392d467f8ef20bd2c8fc85c7', 'request_id': 'req-0623d65f-66a9-417c-a50f-c3d28de8b575'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:03.487019 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0623d65f-66a9-417c-a50f-c3d28de8b575 admin admin] The token cache is being invalidate because project 68b9e048392d467f8ef20bd2c8fc85c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:03.487311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0623d65f-66a9-417c-a50f-c3d28de8b575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:03.487977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0623d65f-66a9-417c-a50f-c3d28de8b575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:03.496854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1254/2545] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:03 2022] DELETE /identity/v3/projects/68b9e048392d467f8ef20bd2c8fc85c7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:03.498725 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f5e07-7f6f-4206-8d09-ae8e9b7962cc None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:03.500121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f5e07-7f6f-4206-8d09-ae8e9b7962cc None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:03.500327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f5e07-7f6f-4206-8d09-ae8e9b7962cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:03.527102 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-2d7f5e07-7f6f-4206-8d09-ae8e9b7962cc None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:03.528692 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2d7f5e07-7f6f-4206-8d09-ae8e9b7962cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:03.594833 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1292/2546] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:03.602500 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1732a0f-3663-48de-a86c-80e9c238e8eb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:03.602790 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1732a0f-3663-48de-a86c-80e9c238e8eb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:03.603063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1732a0f-3663-48de-a86c-80e9c238e8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:03.622434 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d1732a0f-3663-48de-a86c-80e9c238e8eb None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:03.623643 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d1732a0f-3663-48de-a86c-80e9c238e8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:03.631572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1255/2547] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:03.636759 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f5e07-7f6f-4206-8d09-ae8e9b7962cc None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:03.646858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f5e07-7f6f-4206-8d09-ae8e9b7962cc None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:03.651675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced2447-31de-4785-b1a3-ac5a2f6568f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:03.652326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced2447-31de-4785-b1a3-ac5a2f6568f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:03.652611 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced2447-31de-4785-b1a3-ac5a2f6568f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:03.652885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced2447-31de-4785-b1a3-ac5a2f6568f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:03.653648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced2447-31de-4785-b1a3-ac5a2f6568f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:37:03.653957 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced2447-31de-4785-b1a3-ac5a2f6568f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:03.655559 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced2447-31de-4785-b1a3-ac5a2f6568f5 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:03.667849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1293/2548] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:37:03.672160 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1732a0f-3663-48de-a86c-80e9c238e8eb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:03.681420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1732a0f-3663-48de-a86c-80e9c238e8eb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:03.686704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b270f40-095f-4a6f-ac9b-f95541d4efd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:03.687357 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b270f40-095f-4a6f-ac9b-f95541d4efd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:03.687637 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b270f40-095f-4a6f-ac9b-f95541d4efd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:03.687914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b270f40-095f-4a6f-ac9b-f95541d4efd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:03.688602 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b270f40-095f-4a6f-ac9b-f95541d4efd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:03.690516 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b270f40-095f-4a6f-ac9b-f95541d4efd7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:03.719384 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1256/2549] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:03 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:03.720708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced2447-31de-4785-b1a3-ac5a2f6568f5 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:03.731475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced2447-31de-4785-b1a3-ac5a2f6568f5 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:03.792435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a1f7d-e4d4-4324-a783-daedc1ec8591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:03.793050 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a1f7d-e4d4-4324-a783-daedc1ec8591 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:03.793378 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a1f7d-e4d4-4324-a783-daedc1ec8591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:03.793654 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a1f7d-e4d4-4324-a783-daedc1ec8591 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:03.794340 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a1f7d-e4d4-4324-a783-daedc1ec8591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:03.796268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a1f7d-e4d4-4324-a783-daedc1ec8591 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:03.820993 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1294/2550] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:03 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:03.822184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b270f40-095f-4a6f-ac9b-f95541d4efd7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:03.836367 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b270f40-095f-4a6f-ac9b-f95541d4efd7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:03.842664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820a7de-4908-4264-b763-c46ed1cf4010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:03.843216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820a7de-4908-4264-b763-c46ed1cf4010 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:03.843559 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820a7de-4908-4264-b763-c46ed1cf4010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:03.843870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820a7de-4908-4264-b763-c46ed1cf4010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:03.844658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820a7de-4908-4264-b763-c46ed1cf4010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:03.846356 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820a7de-4908-4264-b763-c46ed1cf4010 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:03.858497 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1257/2551] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:37:03.862611 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a1f7d-e4d4-4324-a783-daedc1ec8591 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:03.874556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a1f7d-e4d4-4324-a783-daedc1ec8591 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:03.879965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9e8a6-681a-44f5-83c5-1c5cd48eeb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:03.880805 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9e8a6-681a-44f5-83c5-1c5cd48eeb34 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:03.881090 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9e8a6-681a-44f5-83c5-1c5cd48eeb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:03.881419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9e8a6-681a-44f5-83c5-1c5cd48eeb34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:03.882193 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9e8a6-681a-44f5-83c5-1c5cd48eeb34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:03.883861 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9e8a6-681a-44f5-83c5-1c5cd48eeb34 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:03.886432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:03.896157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1295/2552] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:03.900793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820a7de-4908-4264-b763-c46ed1cf4010 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:03.915652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820a7de-4908-4264-b763-c46ed1cf4010 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:03.921297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778c9e2-e895-4363-9dd1-3021fb4d3a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:03.922149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778c9e2-e895-4363-9dd1-3021fb4d3a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:03.922435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778c9e2-e895-4363-9dd1-3021fb4d3a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:03.922718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778c9e2-e895-4363-9dd1-3021fb4d3a4c admin admin] PATH_INFO: `/v3/projects/1e5ade04ee1e46a3b56135bef158e1d1/users/e07e9457e5f94eae9381cd236b049504/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:03.944350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778c9e2-e895-4363-9dd1-3021fb4d3a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e5ade04ee1e46a3b56135bef158e1d1, user_id=e07e9457e5f94eae9381cd236b049504, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:03.947220 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778c9e2-e895-4363-9dd1-3021fb4d3a4c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:03.960614 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1258/2553] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:03 2022] PUT /identity/v3/projects/1e5ade04ee1e46a3b56135bef158e1d1/users/e07e9457e5f94eae9381cd236b049504/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:03.971175 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee8ea9-31de-4128-81cf-ad9d3443c1d2 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:03.971479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee8ea9-31de-4128-81cf-ad9d3443c1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:03.971755 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee8ea9-31de-4128-81cf-ad9d3443c1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:04.001812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-8eee8ea9-31de-4128-81cf-ad9d3443c1d2 None None] MFA Rules not processed for user `e07e9457e5f94eae9381cd236b049504`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:04.003040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-8eee8ea9-31de-4128-81cf-ad9d3443c1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:04.080404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1296/2554] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:03 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:04.093163 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778c9e2-e895-4363-9dd1-3021fb4d3a4c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:04.094772 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2778c9e2-e895-4363-9dd1-3021fb4d3a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:04.159417 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778c9e2-e895-4363-9dd1-3021fb4d3a4c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:04.163676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c0f03-a525-436d-a235-959052467e86 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '215c6f6f1aad486f85ce20590d889cbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:04.164437 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c0f03-a525-436d-a235-959052467e86 service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:04.164789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c0f03-a525-436d-a235-959052467e86 service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:04.165157 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c0f03-a525-436d-a235-959052467e86 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:04.176464 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c0f03-a525-436d-a235-959052467e86 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:04.177983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c0f03-a525-436d-a235-959052467e86 service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:04.181738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:04.189562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1259/2555] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:04.767922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee8ea9-31de-4128-81cf-ad9d3443c1d2 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:04.771793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-8eee8ea9-31de-4128-81cf-ad9d3443c1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:04.858967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee8ea9-31de-4128-81cf-ad9d3443c1d2 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:04.863779 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba6d22-43ce-4a6b-a85a-5fda4c4c7575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:04.864656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba6d22-43ce-4a6b-a85a-5fda4c4c7575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:04.864937 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba6d22-43ce-4a6b-a85a-5fda4c4c7575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:04.865249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba6d22-43ce-4a6b-a85a-5fda4c4c7575 admin admin] PATH_INFO: `/v3/users/5507329a6bb84cfbbdac99b85f346ab8` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:04.866436 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba6d22-43ce-4a6b-a85a-5fda4c4c7575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5507329a6bb84cfbbdac99b85f346ab8)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:04.868427 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba6d22-43ce-4a6b-a85a-5fda4c4c7575 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:04.915769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-99ba6d22-43ce-4a6b-a85a-5fda4c4c7575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5507329a6bb84cfbbdac99b85f346ab8'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:04.927180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-99ba6d22-43ce-4a6b-a85a-5fda4c4c7575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5507329a6bb84cfbbdac99b85f346ab8'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:04.927835 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-99ba6d22-43ce-4a6b-a85a-5fda4c4c7575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5507329a6bb84cfbbdac99b85f346ab8'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:04.935717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-99ba6d22-43ce-4a6b-a85a-5fda4c4c7575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5507329a6bb84cfbbdac99b85f346ab8'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:04.936388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-99ba6d22-43ce-4a6b-a85a-5fda4c4c7575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5507329a6bb84cfbbdac99b85f346ab8'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:04.949204 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1297/2556] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:04 2022] DELETE /identity/v3/users/5507329a6bb84cfbbdac99b85f346ab8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:05.132034 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c0f03-a525-436d-a235-959052467e86 service glance] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:05.146443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c0f03-a525-436d-a235-959052467e86 service glance] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:05.207853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34224fe-289a-4a22-84d4-5c7bfc486990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:05.208713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34224fe-289a-4a22-84d4-5c7bfc486990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:05.208992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34224fe-289a-4a22-84d4-5c7bfc486990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:05.209311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34224fe-289a-4a22-84d4-5c7bfc486990 admin admin] PATH_INFO: `/v3/projects/af417e926a2d4d328a37fbb7baedbe78` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:05.210493 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34224fe-289a-4a22-84d4-5c7bfc486990 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af417e926a2d4d328a37fbb7baedbe78)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:05.212395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34224fe-289a-4a22-84d4-5c7bfc486990 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:05.264554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b34224fe-289a-4a22-84d4-5c7bfc486990 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af417e926a2d4d328a37fbb7baedbe78', 'request_id': 'req-b34224fe-289a-4a22-84d4-5c7bfc486990'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:05.272523 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b34224fe-289a-4a22-84d4-5c7bfc486990 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af417e926a2d4d328a37fbb7baedbe78', 'request_id': 'req-b34224fe-289a-4a22-84d4-5c7bfc486990'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:05.279464 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b34224fe-289a-4a22-84d4-5c7bfc486990 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af417e926a2d4d328a37fbb7baedbe78', 'request_id': 'req-b34224fe-289a-4a22-84d4-5c7bfc486990'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:05.293089 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b34224fe-289a-4a22-84d4-5c7bfc486990 admin admin] The token cache is being invalidate because project af417e926a2d4d328a37fbb7baedbe78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:05.293410 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b34224fe-289a-4a22-84d4-5c7bfc486990 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:05.294058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b34224fe-289a-4a22-84d4-5c7bfc486990 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:05.295719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1260/2557] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:05 2022] DELETE /identity/v3/projects/af417e926a2d4d328a37fbb7baedbe78 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:05.300908 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c926b5c-1411-4b25-9197-ec392e35e3e4 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:05.301383 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c926b5c-1411-4b25-9197-ec392e35e3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:05.301681 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c926b5c-1411-4b25-9197-ec392e35e3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:05.316241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:05.324538 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-5c926b5c-1411-4b25-9197-ec392e35e3e4 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:05.326336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5c926b5c-1411-4b25-9197-ec392e35e3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:05.392897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1298/2558] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:05.401255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a380eb8-6403-4be6-a97b-8c92110599b7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:05.402168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a380eb8-6403-4be6-a97b-8c92110599b7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:05.402452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a380eb8-6403-4be6-a97b-8c92110599b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:05.423049 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0a380eb8-6403-4be6-a97b-8c92110599b7 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:05.424462 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0a380eb8-6403-4be6-a97b-8c92110599b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:05.433093 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1261/2559] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:05.438095 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c926b5c-1411-4b25-9197-ec392e35e3e4 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:05.447995 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c926b5c-1411-4b25-9197-ec392e35e3e4 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:05.461301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d9861-48fd-41b4-a250-93a95eadbe9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:05.461838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d9861-48fd-41b4-a250-93a95eadbe9a admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:05.462153 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d9861-48fd-41b4-a250-93a95eadbe9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:05.462435 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d9861-48fd-41b4-a250-93a95eadbe9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:05.463290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d9861-48fd-41b4-a250-93a95eadbe9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:37:05.463658 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d9861-48fd-41b4-a250-93a95eadbe9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:05.466523 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d9861-48fd-41b4-a250-93a95eadbe9a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:05.481296 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1299/2560] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:37:05.486538 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a380eb8-6403-4be6-a97b-8c92110599b7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:05.496018 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a380eb8-6403-4be6-a97b-8c92110599b7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:05.500670 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b8ef1-fb74-4dd7-9349-9fe80c3c03b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:05.501351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b8ef1-fb74-4dd7-9349-9fe80c3c03b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:05.501633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b8ef1-fb74-4dd7-9349-9fe80c3c03b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:05.501904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b8ef1-fb74-4dd7-9349-9fe80c3c03b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:05.502610 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b8ef1-fb74-4dd7-9349-9fe80c3c03b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:05.504734 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b8ef1-fb74-4dd7-9349-9fe80c3c03b2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:05.534379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1262/2561] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:05 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:05.538521 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d9861-48fd-41b4-a250-93a95eadbe9a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:05.550180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d9861-48fd-41b4-a250-93a95eadbe9a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:05.608197 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b8ef1-fb74-4dd7-9349-9fe80c3c03b2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:05.609956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f20b8ef1-fb74-4dd7-9349-9fe80c3c03b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:05.610258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d90036-3244-4f1f-943e-1f9376763e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:05.610956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d90036-3244-4f1f-943e-1f9376763e2d admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:05.611246 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d90036-3244-4f1f-943e-1f9376763e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:05.611527 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d90036-3244-4f1f-943e-1f9376763e2d admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:05.612234 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d90036-3244-4f1f-943e-1f9376763e2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:05.614201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d90036-3244-4f1f-943e-1f9376763e2d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:05.623609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b8ef1-fb74-4dd7-9349-9fe80c3c03b2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:05.628784 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f80e5-edd2-42ec-b9a5-c28d871cabb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:05.629720 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f80e5-edd2-42ec-b9a5-c28d871cabb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:05.629986 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f80e5-edd2-42ec-b9a5-c28d871cabb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:05.630270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f80e5-edd2-42ec-b9a5-c28d871cabb3 admin admin] PATH_INFO: `/v3/users/94b33546c2fd4212a2f05566862876ea` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:05.631547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f80e5-edd2-42ec-b9a5-c28d871cabb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94b33546c2fd4212a2f05566862876ea)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:05.633532 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f80e5-edd2-42ec-b9a5-c28d871cabb3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:05.639882 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1300/2562] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:05 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:37:05.641547 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d90036-3244-4f1f-943e-1f9376763e2d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:05.653811 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d90036-3244-4f1f-943e-1f9376763e2d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:05.659254 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f70a85-aa01-4b90-a215-e5a397f890e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:05.659917 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f70a85-aa01-4b90-a215-e5a397f890e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:05.660202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f70a85-aa01-4b90-a215-e5a397f890e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:05.660500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f70a85-aa01-4b90-a215-e5a397f890e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:05.661304 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f70a85-aa01-4b90-a215-e5a397f890e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:05.662960 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f70a85-aa01-4b90-a215-e5a397f890e4 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:05.680254 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1301/2563] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:05.681639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f70a85-aa01-4b90-a215-e5a397f890e4 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:05.684450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d34f80e5-edd2-42ec-b9a5-c28d871cabb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94b33546c2fd4212a2f05566862876ea'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:05.689837 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:05.691624 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f70a85-aa01-4b90-a215-e5a397f890e4 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:05.697198 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257adec-e7c2-4890-b8e9-b96c84db8858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:05.697851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257adec-e7c2-4890-b8e9-b96c84db8858 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:05.698227 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257adec-e7c2-4890-b8e9-b96c84db8858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:05.698544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257adec-e7c2-4890-b8e9-b96c84db8858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:05.698766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d34f80e5-edd2-42ec-b9a5-c28d871cabb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94b33546c2fd4212a2f05566862876ea'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:05.699468 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257adec-e7c2-4890-b8e9-b96c84db8858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:05.699582 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d34f80e5-edd2-42ec-b9a5-c28d871cabb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94b33546c2fd4212a2f05566862876ea'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:05.700314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d34f80e5-edd2-42ec-b9a5-c28d871cabb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94b33546c2fd4212a2f05566862876ea'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:05.701287 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257adec-e7c2-4890-b8e9-b96c84db8858 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:05.708819 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d34f80e5-edd2-42ec-b9a5-c28d871cabb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94b33546c2fd4212a2f05566862876ea'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:05.719971 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1302/2564] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:05.720996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257adec-e7c2-4890-b8e9-b96c84db8858 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:05.722750 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2257adec-e7c2-4890-b8e9-b96c84db8858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:05.723283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1263/2565] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:05 2022] DELETE /identity/v3/users/94b33546c2fd4212a2f05566862876ea => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:05.727539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075228c5-cfc4-4d44-bf26-80d190086d5b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:05.727834 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075228c5-cfc4-4d44-bf26-80d190086d5b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:05.728114 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075228c5-cfc4-4d44-bf26-80d190086d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:05.736758 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257adec-e7c2-4890-b8e9-b96c84db8858 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:05.748343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-075228c5-cfc4-4d44-bf26-80d190086d5b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:05.749678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-075228c5-cfc4-4d44-bf26-80d190086d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:05.806070 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb04627-922f-404d-9aa7-fb2ef68947ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:05.806977 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb04627-922f-404d-9aa7-fb2ef68947ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:05.807261 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb04627-922f-404d-9aa7-fb2ef68947ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:05.807766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb04627-922f-404d-9aa7-fb2ef68947ec admin admin] PATH_INFO: `/v3/projects/bc416573f002485db513e9a7ccc04940/users/397595d3539040808cd88a66855c6710/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:05.812376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:05.833960 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1264/2566] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:05.836264 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb04627-922f-404d-9aa7-fb2ef68947ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc416573f002485db513e9a7ccc04940, user_id=397595d3539040808cd88a66855c6710, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:05.839536 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb04627-922f-404d-9aa7-fb2ef68947ec admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:05.844003 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075228c5-cfc4-4d44-bf26-80d190086d5b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:05.845495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-075228c5-cfc4-4d44-bf26-80d190086d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:05.857573 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1303/2567] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:05 2022] PUT /identity/v3/projects/bc416573f002485db513e9a7ccc04940/users/397595d3539040808cd88a66855c6710/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:05.869368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93158985-b263-433f-9521-88aa22334b7a None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:05.869653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93158985-b263-433f-9521-88aa22334b7a None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:05.869951 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93158985-b263-433f-9521-88aa22334b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:05.899386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-93158985-b263-433f-9521-88aa22334b7a None None] MFA Rules not processed for user `397595d3539040808cd88a66855c6710`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:05.900829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-93158985-b263-433f-9521-88aa22334b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:05.910108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075228c5-cfc4-4d44-bf26-80d190086d5b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:05.961216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dfeda-4890-4086-9ae1-47d1dfbd74aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:05.962121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dfeda-4890-4086-9ae1-47d1dfbd74aa service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:05.962402 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dfeda-4890-4086-9ae1-47d1dfbd74aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:05.962679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dfeda-4890-4086-9ae1-47d1dfbd74aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:05.970995 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1304/2568] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:05.975516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93158985-b263-433f-9521-88aa22334b7a None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:06.012409 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:06.036210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77dfeda-4890-4086-9ae1-47d1dfbd74aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:06.037691 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77dfeda-4890-4086-9ae1-47d1dfbd74aa service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:06.046026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93158985-b263-433f-9521-88aa22334b7a None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:06.049455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb00c8e-527e-41f3-b590-904b158a0440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:06.050207 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1265/2569] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:37:06.050340 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb00c8e-527e-41f3-b590-904b158a0440 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.050656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb00c8e-527e-41f3-b590-904b158a0440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.051027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb00c8e-527e-41f3-b590-904b158a0440 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.051878 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb00c8e-527e-41f3-b590-904b158a0440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:06.053918 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb00c8e-527e-41f3-b590-904b158a0440 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:06.084016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1305/2570] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:05 2022] POST /identity/v3/projects => generated 363 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:37:06.085792 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dfeda-4890-4086-9ae1-47d1dfbd74aa service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:06.099301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dfeda-4890-4086-9ae1-47d1dfbd74aa service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:06.164849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d9ce8-cdc8-4378-8387-77bf87d6724b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:06.165677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d9ce8-cdc8-4378-8387-77bf87d6724b admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.165858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d9ce8-cdc8-4378-8387-77bf87d6724b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.166162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d9ce8-cdc8-4378-8387-77bf87d6724b admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.166903 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d9ce8-cdc8-4378-8387-77bf87d6724b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:06.169003 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d9ce8-cdc8-4378-8387-77bf87d6724b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:06.192874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1266/2571] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:06 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:37:06.194479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb00c8e-527e-41f3-b590-904b158a0440 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:06.206541 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb00c8e-527e-41f3-b590-904b158a0440 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:06.211238 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d3ff9-e383-4098-94a3-3a1a3dd96b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:06.211919 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d3ff9-e383-4098-94a3-3a1a3dd96b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.212247 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d3ff9-e383-4098-94a3-3a1a3dd96b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.212467 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d3ff9-e383-4098-94a3-3a1a3dd96b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.213392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d3ff9-e383-4098-94a3-3a1a3dd96b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:06.215852 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d3ff9-e383-4098-94a3-3a1a3dd96b96 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:06.227269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1306/2572] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:06.231800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d9ce8-cdc8-4378-8387-77bf87d6724b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:06.243016 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d9ce8-cdc8-4378-8387-77bf87d6724b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:06.249898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e443b8-dbaf-40e2-b6d3-283314e3dbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:06.251137 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e443b8-dbaf-40e2-b6d3-283314e3dbe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.251137 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e443b8-dbaf-40e2-b6d3-283314e3dbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.251488 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e443b8-dbaf-40e2-b6d3-283314e3dbe3 admin admin] PATH_INFO: `/v3/projects/175851ed124349349a3706ceac2b448d/users/3f50f83e1a4a449fb79c83415abb7f2c/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.276317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e443b8-dbaf-40e2-b6d3-283314e3dbe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175851ed124349349a3706ceac2b448d, user_id=3f50f83e1a4a449fb79c83415abb7f2c, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:06.279045 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e443b8-dbaf-40e2-b6d3-283314e3dbe3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:06.294520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1267/2573] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:06 2022] PUT /identity/v3/projects/175851ed124349349a3706ceac2b448d/users/3f50f83e1a4a449fb79c83415abb7f2c/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:06.298544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d3ff9-e383-4098-94a3-3a1a3dd96b96 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:06.309070 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d3ff9-e383-4098-94a3-3a1a3dd96b96 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:06.362780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e443b8-dbaf-40e2-b6d3-283314e3dbe3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:06.362938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:06.364537 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-03e443b8-dbaf-40e2-b6d3-283314e3dbe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:06.381863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f35a3-42a1-407b-852b-6c8a9ffd298e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:06.382578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f35a3-42a1-407b-852b-6c8a9ffd298e admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.382915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f35a3-42a1-407b-852b-6c8a9ffd298e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.383197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f35a3-42a1-407b-852b-6c8a9ffd298e admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.384027 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f35a3-42a1-407b-852b-6c8a9ffd298e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:06.391561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f35a3-42a1-407b-852b-6c8a9ffd298e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:06.410788 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1307/2574] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:06.416425 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f35a3-42a1-407b-852b-6c8a9ffd298e admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:06.428066 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f35a3-42a1-407b-852b-6c8a9ffd298e admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:06.433847 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87077f2-fc8c-4794-9533-6e031ab0b967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:06.434887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87077f2-fc8c-4794-9533-6e031ab0b967 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.435169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87077f2-fc8c-4794-9533-6e031ab0b967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.435451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87077f2-fc8c-4794-9533-6e031ab0b967 admin admin] PATH_INFO: `/v3/projects/175851ed124349349a3706ceac2b448d/users/3f50f83e1a4a449fb79c83415abb7f2c/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.437513 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87077f2-fc8c-4794-9533-6e031ab0b967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175851ed124349349a3706ceac2b448d, user_id=3f50f83e1a4a449fb79c83415abb7f2c, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:06.440301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87077f2-fc8c-4794-9533-6e031ab0b967 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:06.449452 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e443b8-dbaf-40e2-b6d3-283314e3dbe3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:06.452000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1308/2575] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:06 2022] PUT /identity/v3/projects/175851ed124349349a3706ceac2b448d/users/3f50f83e1a4a449fb79c83415abb7f2c/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:06.462397 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf944d-1818-4f8a-aede-6439849f26a9 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.462747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf944d-1818-4f8a-aede-6439849f26a9 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.463087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf944d-1818-4f8a-aede-6439849f26a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.492550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-3dcf944d-1818-4f8a-aede-6439849f26a9 None None] MFA Rules not processed for user `3f50f83e1a4a449fb79c83415abb7f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:06.493687 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3dcf944d-1818-4f8a-aede-6439849f26a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:06.499158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:06.512806 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760e9e0-68c3-4ea5-9947-081e835d93de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:06.513735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760e9e0-68c3-4ea5-9947-081e835d93de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.514020 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760e9e0-68c3-4ea5-9947-081e835d93de admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.514296 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760e9e0-68c3-4ea5-9947-081e835d93de admin admin] PATH_INFO: `/v3/projects/5bb5042a3bb24a8faf2f9e131cfbfec4` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.515500 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760e9e0-68c3-4ea5-9947-081e835d93de admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bb5042a3bb24a8faf2f9e131cfbfec4)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:06.517487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760e9e0-68c3-4ea5-9947-081e835d93de admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:06.570324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d760e9e0-68c3-4ea5-9947-081e835d93de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bb5042a3bb24a8faf2f9e131cfbfec4', 'request_id': 'req-d760e9e0-68c3-4ea5-9947-081e835d93de'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:06.579033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d760e9e0-68c3-4ea5-9947-081e835d93de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bb5042a3bb24a8faf2f9e131cfbfec4', 'request_id': 'req-d760e9e0-68c3-4ea5-9947-081e835d93de'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:06.585445 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1309/2576] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:06 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:06.585616 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d760e9e0-68c3-4ea5-9947-081e835d93de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bb5042a3bb24a8faf2f9e131cfbfec4', 'request_id': 'req-d760e9e0-68c3-4ea5-9947-081e835d93de'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:06.593272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf944d-1818-4f8a-aede-6439849f26a9 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:06.594769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3dcf944d-1818-4f8a-aede-6439849f26a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:06.599186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d760e9e0-68c3-4ea5-9947-081e835d93de admin admin] The token cache is being invalidate because project 5bb5042a3bb24a8faf2f9e131cfbfec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:06.599483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d760e9e0-68c3-4ea5-9947-081e835d93de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:06.600785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d760e9e0-68c3-4ea5-9947-081e835d93de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:06.602427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1268/2577] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:06 2022] DELETE /identity/v3/projects/5bb5042a3bb24a8faf2f9e131cfbfec4 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:06.607480 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb88ed5-4e6a-498b-8991-cad5fe733e34 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.607769 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb88ed5-4e6a-498b-8991-cad5fe733e34 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.608044 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb88ed5-4e6a-498b-8991-cad5fe733e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.626551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-beb88ed5-4e6a-498b-8991-cad5fe733e34 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:06.627789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-beb88ed5-4e6a-498b-8991-cad5fe733e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:06.662085 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf944d-1818-4f8a-aede-6439849f26a9 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:06.694788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1269/2578] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:06.702685 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0b28d-6420-4809-bfda-e9e76aa00973 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.702975 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0b28d-6420-4809-bfda-e9e76aa00973 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.703251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0b28d-6420-4809-bfda-e9e76aa00973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.717748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51380e-668f-4cdc-ba97-74c23bd8a1d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:06.718432 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51380e-668f-4cdc-ba97-74c23bd8a1d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.718713 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51380e-668f-4cdc-ba97-74c23bd8a1d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.718990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51380e-668f-4cdc-ba97-74c23bd8a1d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.720772 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2a51380e-668f-4cdc-ba97-74c23bd8a1d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:06.720905 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-83e0b28d-6420-4809-bfda-e9e76aa00973 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:06.721988 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-83e0b28d-6420-4809-bfda-e9e76aa00973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:06.730443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1270/2579] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:06.734590 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0b28d-6420-4809-bfda-e9e76aa00973 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:06.744707 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0b28d-6420-4809-bfda-e9e76aa00973 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:06.751030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0f40b-1bd3-4db2-82da-206daef7f591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:06.751743 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0f40b-1bd3-4db2-82da-206daef7f591 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.752032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0f40b-1bd3-4db2-82da-206daef7f591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.752317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0f40b-1bd3-4db2-82da-206daef7f591 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.753064 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0f40b-1bd3-4db2-82da-206daef7f591 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:37:06.753404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0f40b-1bd3-4db2-82da-206daef7f591 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:06.755048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0f40b-1bd3-4db2-82da-206daef7f591 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:06.757413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:06.768539 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1271/2580] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:37:06.772924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0f40b-1bd3-4db2-82da-206daef7f591 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:06.783562 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0f40b-1bd3-4db2-82da-206daef7f591 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:06.788257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b638fd-ebce-48f3-b43e-e481e65ca284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:06.788895 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b638fd-ebce-48f3-b43e-e481e65ca284 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.789212 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b638fd-ebce-48f3-b43e-e481e65ca284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.789494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b638fd-ebce-48f3-b43e-e481e65ca284 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.790186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b638fd-ebce-48f3-b43e-e481e65ca284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:06.792694 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b638fd-ebce-48f3-b43e-e481e65ca284 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:06.798376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51380e-668f-4cdc-ba97-74c23bd8a1d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:06.800201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51380e-668f-4cdc-ba97-74c23bd8a1d2 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:06.812578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1310/2581] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:06 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:37:06.823260 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1272/2582] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:06 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:37:06.827745 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51380e-668f-4cdc-ba97-74c23bd8a1d2 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:06.839405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51380e-668f-4cdc-ba97-74c23bd8a1d2 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:06.890974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b638fd-ebce-48f3-b43e-e481e65ca284 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:06.892819 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-02b638fd-ebce-48f3-b43e-e481e65ca284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:06.912764 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863a14b-167c-4651-8c62-8504374ff460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:06.913476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863a14b-167c-4651-8c62-8504374ff460 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.913764 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863a14b-167c-4651-8c62-8504374ff460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.914041 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863a14b-167c-4651-8c62-8504374ff460 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.914744 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863a14b-167c-4651-8c62-8504374ff460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:06.916688 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863a14b-167c-4651-8c62-8504374ff460 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:06.938502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1311/2583] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:06 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:06.942429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863a14b-167c-4651-8c62-8504374ff460 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:06.947079 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:06.957667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863a14b-167c-4651-8c62-8504374ff460 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:06.962496 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957bd22-54fa-42ff-95d4-aac921fd4ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:06.963170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957bd22-54fa-42ff-95d4-aac921fd4ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.963459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957bd22-54fa-42ff-95d4-aac921fd4ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.963782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957bd22-54fa-42ff-95d4-aac921fd4ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.964536 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957bd22-54fa-42ff-95d4-aac921fd4ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:06.964759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b638fd-ebce-48f3-b43e-e481e65ca284 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:06.966185 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957bd22-54fa-42ff-95d4-aac921fd4ff6 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:06.968356 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3c908-96bb-438a-9eda-8ba28140cdb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:06.969026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3c908-96bb-438a-9eda-8ba28140cdb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.969336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3c908-96bb-438a-9eda-8ba28140cdb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.969609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3c908-96bb-438a-9eda-8ba28140cdb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.971374 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-02e3c908-96bb-438a-9eda-8ba28140cdb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:06.979865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1312/2584] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:37:06.981181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957bd22-54fa-42ff-95d4-aac921fd4ff6 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:06.990956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957bd22-54fa-42ff-95d4-aac921fd4ff6 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:06.996906 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca90f36-01e5-47b9-b88d-53dfa1a16835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:06.997627 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca90f36-01e5-47b9-b88d-53dfa1a16835 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:06.997935 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca90f36-01e5-47b9-b88d-53dfa1a16835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:06.998217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca90f36-01e5-47b9-b88d-53dfa1a16835 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:06.999003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca90f36-01e5-47b9-b88d-53dfa1a16835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:07.001581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca90f36-01e5-47b9-b88d-53dfa1a16835 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:07.013950 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1313/2585] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:37:07.018061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca90f36-01e5-47b9-b88d-53dfa1a16835 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:07.029471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca90f36-01e5-47b9-b88d-53dfa1a16835 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:07.034726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c07fd8-8839-4386-9a85-6b8e9de52e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:07.035655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c07fd8-8839-4386-9a85-6b8e9de52e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:07.035937 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c07fd8-8839-4386-9a85-6b8e9de52e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:07.036210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c07fd8-8839-4386-9a85-6b8e9de52e0a admin admin] PATH_INFO: `/v3/projects/237cfd9d3875450f85e59aa266314621/users/2fe15b9410f148d09dc3c5e5da9e8fc6/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:07.036441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3c908-96bb-438a-9eda-8ba28140cdb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:07.037994 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3c908-96bb-438a-9eda-8ba28140cdb6 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:07.048237 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1273/2586] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:37:07.053722 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c07fd8-8839-4386-9a85-6b8e9de52e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=237cfd9d3875450f85e59aa266314621, user_id=2fe15b9410f148d09dc3c5e5da9e8fc6, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:07.056651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c07fd8-8839-4386-9a85-6b8e9de52e0a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:07.076585 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1314/2587] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:07 2022] PUT /identity/v3/projects/237cfd9d3875450f85e59aa266314621/users/2fe15b9410f148d09dc3c5e5da9e8fc6/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:07.147841 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13205a6c-9aab-42da-ab8d-78197dbcee3e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:07.148133 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13205a6c-9aab-42da-ab8d-78197dbcee3e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:07.148418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13205a6c-9aab-42da-ab8d-78197dbcee3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:07.180487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-13205a6c-9aab-42da-ab8d-78197dbcee3e None None] MFA Rules not processed for user `2fe15b9410f148d09dc3c5e5da9e8fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:07.181900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-13205a6c-9aab-42da-ab8d-78197dbcee3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:07.266717 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1274/2588] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:07.283737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c07fd8-8839-4386-9a85-6b8e9de52e0a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:07.285516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-39c07fd8-8839-4386-9a85-6b8e9de52e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:07.476904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c07fd8-8839-4386-9a85-6b8e9de52e0a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:07.481625 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a4fba-e26f-47e6-8c60-fcadd4f0a807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:07.482302 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a4fba-e26f-47e6-8c60-fcadd4f0a807 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:07.482589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a4fba-e26f-47e6-8c60-fcadd4f0a807 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:07.482867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a4fba-e26f-47e6-8c60-fcadd4f0a807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:07.492240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a4fba-e26f-47e6-8c60-fcadd4f0a807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:07.493785 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a4fba-e26f-47e6-8c60-fcadd4f0a807 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:07.503479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1315/2589] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:17.263876 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=62 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:17.499000 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=92,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:25.971558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13205a6c-9aab-42da-ab8d-78197dbcee3e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:25.973063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-13205a6c-9aab-42da-ab8d-78197dbcee3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:26.044205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13205a6c-9aab-42da-ab8d-78197dbcee3e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:26.048877 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09356e91-393b-4990-91ee-0b345ecd6cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:26.049802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09356e91-393b-4990-91ee-0b345ecd6cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:26.050107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09356e91-393b-4990-91ee-0b345ecd6cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:26.050381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09356e91-393b-4990-91ee-0b345ecd6cae admin admin] PATH_INFO: `/v3/users/397595d3539040808cd88a66855c6710` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:26.051723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09356e91-393b-4990-91ee-0b345ecd6cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=397595d3539040808cd88a66855c6710)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:26.054200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09356e91-393b-4990-91ee-0b345ecd6cae admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:26.107199 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-09356e91-393b-4990-91ee-0b345ecd6cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '397595d3539040808cd88a66855c6710'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:26.120426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-09356e91-393b-4990-91ee-0b345ecd6cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '397595d3539040808cd88a66855c6710'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:26.121336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-09356e91-393b-4990-91ee-0b345ecd6cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '397595d3539040808cd88a66855c6710'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:26.122077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-09356e91-393b-4990-91ee-0b345ecd6cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '397595d3539040808cd88a66855c6710'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:26.130117 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-09356e91-393b-4990-91ee-0b345ecd6cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '397595d3539040808cd88a66855c6710'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:26.145812 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1275/2590] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:25 2022] DELETE /identity/v3/users/397595d3539040808cd88a66855c6710 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:26.150957 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91894fd-e19c-4381-a636-7f0a85cb5c38 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:26.151421 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91894fd-e19c-4381-a636-7f0a85cb5c38 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:26.151685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91894fd-e19c-4381-a636-7f0a85cb5c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:26.170751 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-d91894fd-e19c-4381-a636-7f0a85cb5c38 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:26.171962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d91894fd-e19c-4381-a636-7f0a85cb5c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:26.235451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1316/2591] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:26.245798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09356e91-393b-4990-91ee-0b345ecd6cae admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:26.247372 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-09356e91-393b-4990-91ee-0b345ecd6cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:26.306186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09356e91-393b-4990-91ee-0b345ecd6cae admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:26.310242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d476c0-fbf9-4d18-bcbd-dc1ad4e88bf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:26.310900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d476c0-fbf9-4d18-bcbd-dc1ad4e88bf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:26.311190 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d476c0-fbf9-4d18-bcbd-dc1ad4e88bf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:26.311473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d476c0-fbf9-4d18-bcbd-dc1ad4e88bf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:26.321256 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d476c0-fbf9-4d18-bcbd-dc1ad4e88bf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:26.322569 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d476c0-fbf9-4d18-bcbd-dc1ad4e88bf5 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:26.333314 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1276/2592] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:26.601196 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91894fd-e19c-4381-a636-7f0a85cb5c38 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:26.603400 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d91894fd-e19c-4381-a636-7f0a85cb5c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:26.621324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91894fd-e19c-4381-a636-7f0a85cb5c38 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:26.627854 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28984d96-043b-4a88-b809-68a279bca290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:26.629677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28984d96-043b-4a88-b809-68a279bca290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:26.630008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28984d96-043b-4a88-b809-68a279bca290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:26.630417 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28984d96-043b-4a88-b809-68a279bca290 admin admin] PATH_INFO: `/v3/projects/bc416573f002485db513e9a7ccc04940` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:26.631663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28984d96-043b-4a88-b809-68a279bca290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc416573f002485db513e9a7ccc04940)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:26.633788 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28984d96-043b-4a88-b809-68a279bca290 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:26.696832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-28984d96-043b-4a88-b809-68a279bca290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc416573f002485db513e9a7ccc04940', 'request_id': 'req-28984d96-043b-4a88-b809-68a279bca290'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:26.702893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-28984d96-043b-4a88-b809-68a279bca290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc416573f002485db513e9a7ccc04940', 'request_id': 'req-28984d96-043b-4a88-b809-68a279bca290'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:26.714392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-28984d96-043b-4a88-b809-68a279bca290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc416573f002485db513e9a7ccc04940', 'request_id': 'req-28984d96-043b-4a88-b809-68a279bca290'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:26.728053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-28984d96-043b-4a88-b809-68a279bca290 admin admin] The token cache is being invalidate because project bc416573f002485db513e9a7ccc04940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:26.728434 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-28984d96-043b-4a88-b809-68a279bca290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:26.729045 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-28984d96-043b-4a88-b809-68a279bca290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:26.731550 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1317/2593] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:26 2022] DELETE /identity/v3/projects/bc416573f002485db513e9a7ccc04940 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:26.735771 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d476c0-fbf9-4d18-bcbd-dc1ad4e88bf5 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:26.745587 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d476c0-fbf9-4d18-bcbd-dc1ad4e88bf5 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:26.764204 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:26.805266 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338c4c5-42af-4a09-8175-8569ca4aa1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:26.806152 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338c4c5-42af-4a09-8175-8569ca4aa1ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:26.806436 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338c4c5-42af-4a09-8175-8569ca4aa1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:26.806748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338c4c5-42af-4a09-8175-8569ca4aa1ed admin admin] PATH_INFO: `/v3/users/3f50f83e1a4a449fb79c83415abb7f2c` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:26.807942 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338c4c5-42af-4a09-8175-8569ca4aa1ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f50f83e1a4a449fb79c83415abb7f2c)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:26.810070 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338c4c5-42af-4a09-8175-8569ca4aa1ed admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:26.855896 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5338c4c5-42af-4a09-8175-8569ca4aa1ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f50f83e1a4a449fb79c83415abb7f2c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:26.867446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5338c4c5-42af-4a09-8175-8569ca4aa1ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f50f83e1a4a449fb79c83415abb7f2c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:26.868103 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5338c4c5-42af-4a09-8175-8569ca4aa1ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f50f83e1a4a449fb79c83415abb7f2c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:26.868707 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5338c4c5-42af-4a09-8175-8569ca4aa1ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f50f83e1a4a449fb79c83415abb7f2c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:26.875828 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5338c4c5-42af-4a09-8175-8569ca4aa1ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f50f83e1a4a449fb79c83415abb7f2c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:26.888678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1277/2594] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:26 2022] DELETE /identity/v3/users/3f50f83e1a4a449fb79c83415abb7f2c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:27.154526 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28984d96-043b-4a88-b809-68a279bca290 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:27.156062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-28984d96-043b-4a88-b809-68a279bca290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:27.170377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28984d96-043b-4a88-b809-68a279bca290 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:27.232199 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c7769-7c82-4142-ba66-296931844273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:27.233041 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c7769-7c82-4142-ba66-296931844273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:27.233351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c7769-7c82-4142-ba66-296931844273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:27.233625 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c7769-7c82-4142-ba66-296931844273 admin admin] PATH_INFO: `/v3/projects/175851ed124349349a3706ceac2b448d` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:27.234799 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c7769-7c82-4142-ba66-296931844273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=175851ed124349349a3706ceac2b448d)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:27.236701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c7769-7c82-4142-ba66-296931844273 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:27.270448 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:27.287611 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ca9c7769-7c82-4142-ba66-296931844273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175851ed124349349a3706ceac2b448d', 'request_id': 'req-ca9c7769-7c82-4142-ba66-296931844273'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:27.293568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ca9c7769-7c82-4142-ba66-296931844273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175851ed124349349a3706ceac2b448d', 'request_id': 'req-ca9c7769-7c82-4142-ba66-296931844273'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:27.299625 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ca9c7769-7c82-4142-ba66-296931844273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175851ed124349349a3706ceac2b448d', 'request_id': 'req-ca9c7769-7c82-4142-ba66-296931844273'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:27.311910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ca9c7769-7c82-4142-ba66-296931844273 admin admin] The token cache is being invalidate because project 175851ed124349349a3706ceac2b448d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:27.312198 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ca9c7769-7c82-4142-ba66-296931844273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:27.312826 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-ca9c7769-7c82-4142-ba66-296931844273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:27.314419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1318/2595] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:27 2022] DELETE /identity/v3/projects/175851ed124349349a3706ceac2b448d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:27.319321 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdeb50b-3db5-4c04-882b-3ecebdcf4261 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:27.319800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdeb50b-3db5-4c04-882b-3ecebdcf4261 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:27.319896 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdeb50b-3db5-4c04-882b-3ecebdcf4261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:27.338397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9cdeb50b-3db5-4c04-882b-3ecebdcf4261 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:27.339630 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9cdeb50b-3db5-4c04-882b-3ecebdcf4261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:27.398026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1278/2596] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:27.406811 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70c4cb-0da2-45e5-8eea-bcfc64078839 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:27.407098 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70c4cb-0da2-45e5-8eea-bcfc64078839 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:27.407375 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70c4cb-0da2-45e5-8eea-bcfc64078839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:27.424992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-8f70c4cb-0da2-45e5-8eea-bcfc64078839 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:27.426196 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-8f70c4cb-0da2-45e5-8eea-bcfc64078839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:27.434508 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1319/2597] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:27.438723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdeb50b-3db5-4c04-882b-3ecebdcf4261 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:27.448201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdeb50b-3db5-4c04-882b-3ecebdcf4261 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:27.453371 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be645ac-ddce-4b29-8b78-cf74179a5322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:27.454019 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be645ac-ddce-4b29-8b78-cf74179a5322 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:27.454299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be645ac-ddce-4b29-8b78-cf74179a5322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:27.454572 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be645ac-ddce-4b29-8b78-cf74179a5322 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:27.455322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be645ac-ddce-4b29-8b78-cf74179a5322 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:37:27.455626 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be645ac-ddce-4b29-8b78-cf74179a5322 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:27.457249 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be645ac-ddce-4b29-8b78-cf74179a5322 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:27.469292 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1279/2598] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:37:27.473392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70c4cb-0da2-45e5-8eea-bcfc64078839 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:27.483026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70c4cb-0da2-45e5-8eea-bcfc64078839 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:27.487530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669ec9d-5c10-4fe9-b15e-821715dd66de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:27.488165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669ec9d-5c10-4fe9-b15e-821715dd66de admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:27.488444 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669ec9d-5c10-4fe9-b15e-821715dd66de admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:27.488716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669ec9d-5c10-4fe9-b15e-821715dd66de admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:27.489434 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669ec9d-5c10-4fe9-b15e-821715dd66de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:27.491311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669ec9d-5c10-4fe9-b15e-821715dd66de admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:27.516311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1320/2599] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:27 2022] POST /identity/v3/projects => generated 393 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:27.520708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be645ac-ddce-4b29-8b78-cf74179a5322 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:27.531625 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be645ac-ddce-4b29-8b78-cf74179a5322 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:27.558772 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:27.592418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ec23e-04e2-4f2f-9273-6b426cd71045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:27.593129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ec23e-04e2-4f2f-9273-6b426cd71045 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:27.593446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ec23e-04e2-4f2f-9273-6b426cd71045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:27.593728 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ec23e-04e2-4f2f-9273-6b426cd71045 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:27.594435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ec23e-04e2-4f2f-9273-6b426cd71045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:27.596474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ec23e-04e2-4f2f-9273-6b426cd71045 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:27.616415 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1280/2600] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:27 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:37:27.620519 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669ec9d-5c10-4fe9-b15e-821715dd66de admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:27.630732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669ec9d-5c10-4fe9-b15e-821715dd66de admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:27.635357 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4adc3-758c-4715-b56f-43ad621ec089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:27.636011 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4adc3-758c-4715-b56f-43ad621ec089 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:27.636924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4adc3-758c-4715-b56f-43ad621ec089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:27.637169 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4adc3-758c-4715-b56f-43ad621ec089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:27.638036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4adc3-758c-4715-b56f-43ad621ec089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:27.639631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4adc3-758c-4715-b56f-43ad621ec089 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:27.650167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1321/2601] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:27.654525 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ec23e-04e2-4f2f-9273-6b426cd71045 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:27.664455 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ec23e-04e2-4f2f-9273-6b426cd71045 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:27.669092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d129ed6-aa5f-45b5-979f-463b3223201f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:27.669776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d129ed6-aa5f-45b5-979f-463b3223201f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:27.670063 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d129ed6-aa5f-45b5-979f-463b3223201f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:27.670343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d129ed6-aa5f-45b5-979f-463b3223201f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:27.671074 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d129ed6-aa5f-45b5-979f-463b3223201f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:27.672659 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d129ed6-aa5f-45b5-979f-463b3223201f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:27.682976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1281/2602] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:27.687292 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4adc3-758c-4715-b56f-43ad621ec089 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:27.697915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4adc3-758c-4715-b56f-43ad621ec089 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:27.703118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1f3c7-cd5e-40e4-9640-b2cef560a861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:27.703983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1f3c7-cd5e-40e4-9640-b2cef560a861 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:27.704281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1f3c7-cd5e-40e4-9640-b2cef560a861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:27.704542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1f3c7-cd5e-40e4-9640-b2cef560a861 admin admin] PATH_INFO: `/v3/projects/c8cf22ccdb4644bfb347b3ef48adb4c8/users/2884f1b5b53845e5b11ceb6ce4e5ff47/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:27.719649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1f3c7-cd5e-40e4-9640-b2cef560a861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8cf22ccdb4644bfb347b3ef48adb4c8, user_id=2884f1b5b53845e5b11ceb6ce4e5ff47, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:27.722334 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1f3c7-cd5e-40e4-9640-b2cef560a861 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:27.735170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1322/2603] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:27 2022] PUT /identity/v3/projects/c8cf22ccdb4644bfb347b3ef48adb4c8/users/2884f1b5b53845e5b11ceb6ce4e5ff47/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:27.746560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93773ae-6888-4b4f-8664-3505c9d92ddc None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:27.746866 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93773ae-6888-4b4f-8664-3505c9d92ddc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:27.747144 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93773ae-6888-4b4f-8664-3505c9d92ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:27.774101 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a93773ae-6888-4b4f-8664-3505c9d92ddc None None] MFA Rules not processed for user `2884f1b5b53845e5b11ceb6ce4e5ff47`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:27.775333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a93773ae-6888-4b4f-8664-3505c9d92ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:27.838266 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1282/2604] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:27.843327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1f3c7-cd5e-40e4-9640-b2cef560a861 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:27.904364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1f3c7-cd5e-40e4-9640-b2cef560a861 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:27.908942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d98c4-2890-45b5-b168-ab1155aa353a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:27.909651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d98c4-2890-45b5-b168-ab1155aa353a admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:27.909940 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d98c4-2890-45b5-b168-ab1155aa353a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:27.910221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d98c4-2890-45b5-b168-ab1155aa353a admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:27.910907 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d98c4-2890-45b5-b168-ab1155aa353a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:27.912816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d98c4-2890-45b5-b168-ab1155aa353a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:27.937393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1323/2605] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:27 2022] POST /identity/v3/projects => generated 391 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:27.941621 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93773ae-6888-4b4f-8664-3505c9d92ddc None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:27.953034 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93773ae-6888-4b4f-8664-3505c9d92ddc None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:28.009777 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc76f18-b82a-4843-bd3e-45debc51e949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:28.010445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc76f18-b82a-4843-bd3e-45debc51e949 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:28.010722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc76f18-b82a-4843-bd3e-45debc51e949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:28.010995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc76f18-b82a-4843-bd3e-45debc51e949 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:28.012507 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc76f18-b82a-4843-bd3e-45debc51e949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:28.014437 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc76f18-b82a-4843-bd3e-45debc51e949 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:28.034700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1283/2606] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:27 2022] POST /identity/v3/users => generated 302 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:28.038677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d98c4-2890-45b5-b168-ab1155aa353a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:28.050300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d98c4-2890-45b5-b168-ab1155aa353a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:28.055017 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c067741-eca1-4240-9e16-bee84627c7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:28.055678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c067741-eca1-4240-9e16-bee84627c7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:28.055962 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c067741-eca1-4240-9e16-bee84627c7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:28.056267 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c067741-eca1-4240-9e16-bee84627c7d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:28.057048 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c067741-eca1-4240-9e16-bee84627c7d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:28.059220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c067741-eca1-4240-9e16-bee84627c7d5 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:28.070423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1324/2607] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:28.075646 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc76f18-b82a-4843-bd3e-45debc51e949 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:28.086277 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc76f18-b82a-4843-bd3e-45debc51e949 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:28.091613 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1df0b-e34a-4a0c-8d1b-185c8d71889b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:28.092482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1df0b-e34a-4a0c-8d1b-185c8d71889b admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:28.092763 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1df0b-e34a-4a0c-8d1b-185c8d71889b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:28.093040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1df0b-e34a-4a0c-8d1b-185c8d71889b admin admin] PATH_INFO: `/v3/projects/3f951f1d55f243d385cf37410a704713/users/8265eb8f430c4a12b15d9ccf7e9eb48b/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:28.108476 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1df0b-e34a-4a0c-8d1b-185c8d71889b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f951f1d55f243d385cf37410a704713, user_id=8265eb8f430c4a12b15d9ccf7e9eb48b, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:28.111164 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1df0b-e34a-4a0c-8d1b-185c8d71889b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:28.123708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1284/2608] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:28 2022] PUT /identity/v3/projects/3f951f1d55f243d385cf37410a704713/users/8265eb8f430c4a12b15d9ccf7e9eb48b/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:28.127608 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c067741-eca1-4240-9e16-bee84627c7d5 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:28.137482 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c067741-eca1-4240-9e16-bee84627c7d5 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:28.157536 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:28.193698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877b755-5dd8-4fb7-868a-bd8972f24012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:28.194356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877b755-5dd8-4fb7-868a-bd8972f24012 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:28.194638 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877b755-5dd8-4fb7-868a-bd8972f24012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:28.194910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877b755-5dd8-4fb7-868a-bd8972f24012 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:28.195631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877b755-5dd8-4fb7-868a-bd8972f24012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:28.197236 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877b755-5dd8-4fb7-868a-bd8972f24012 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:28.206723 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1325/2609] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:28.211083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1df0b-e34a-4a0c-8d1b-185c8d71889b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:28.217219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:28.223315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1df0b-e34a-4a0c-8d1b-185c8d71889b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:28.229748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa84a82-48d6-4591-a335-838a7b049c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:28.231475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa84a82-48d6-4591-a335-838a7b049c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:28.231760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa84a82-48d6-4591-a335-838a7b049c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:28.232036 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa84a82-48d6-4591-a335-838a7b049c30 admin admin] PATH_INFO: `/v3/projects/3f951f1d55f243d385cf37410a704713/users/8265eb8f430c4a12b15d9ccf7e9eb48b/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:28.234087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa84a82-48d6-4591-a335-838a7b049c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f951f1d55f243d385cf37410a704713, user_id=8265eb8f430c4a12b15d9ccf7e9eb48b, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:28.236762 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa84a82-48d6-4591-a335-838a7b049c30 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:28.253067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1285/2610] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:28 2022] PUT /identity/v3/projects/3f951f1d55f243d385cf37410a704713/users/8265eb8f430c4a12b15d9ccf7e9eb48b/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:28.258732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9da7a-50d3-41e0-b65c-97e8ce55663d None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:28.259022 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9da7a-50d3-41e0-b65c-97e8ce55663d None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:28.259301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9da7a-50d3-41e0-b65c-97e8ce55663d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:28.285555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-b0a9da7a-50d3-41e0-b65c-97e8ce55663d None None] MFA Rules not processed for user `8265eb8f430c4a12b15d9ccf7e9eb48b`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:28.286753 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b0a9da7a-50d3-41e0-b65c-97e8ce55663d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:28.353192 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1326/2611] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:28.363976 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa84a82-48d6-4591-a335-838a7b049c30 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:28.365821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1fa84a82-48d6-4591-a335-838a7b049c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:28.422829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa84a82-48d6-4591-a335-838a7b049c30 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:28.426393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eef07e-a8ed-4541-b19c-e93aaee87231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:28.427034 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eef07e-a8ed-4541-b19c-e93aaee87231 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:28.427319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eef07e-a8ed-4541-b19c-e93aaee87231 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:28.427598 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eef07e-a8ed-4541-b19c-e93aaee87231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:28.436622 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eef07e-a8ed-4541-b19c-e93aaee87231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:28.438083 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eef07e-a8ed-4541-b19c-e93aaee87231 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:28.447477 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1286/2612] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:28.492471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9da7a-50d3-41e0-b65c-97e8ce55663d None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:28.493862 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b0a9da7a-50d3-41e0-b65c-97e8ce55663d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:28.506066 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9da7a-50d3-41e0-b65c-97e8ce55663d None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:28.509701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb502736-d887-4e7d-b2e9-25b3bac5121b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:28.510342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb502736-d887-4e7d-b2e9-25b3bac5121b service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:28.510624 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb502736-d887-4e7d-b2e9-25b3bac5121b service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:28.510895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb502736-d887-4e7d-b2e9-25b3bac5121b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:28.567665 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb502736-d887-4e7d-b2e9-25b3bac5121b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:28.569282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb502736-d887-4e7d-b2e9-25b3bac5121b service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:28.578882 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1327/2613] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:31.904828 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eef07e-a8ed-4541-b19c-e93aaee87231 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:31.906422 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-08eef07e-a8ed-4541-b19c-e93aaee87231 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:31.974913 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eef07e-a8ed-4541-b19c-e93aaee87231 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:31.979490 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e610782-336e-4df7-8dc4-a58a6defc347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:31.980405 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e610782-336e-4df7-8dc4-a58a6defc347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:31.980678 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e610782-336e-4df7-8dc4-a58a6defc347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:31.980962 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e610782-336e-4df7-8dc4-a58a6defc347 admin admin] PATH_INFO: `/v3/users/2fe15b9410f148d09dc3c5e5da9e8fc6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:31.982231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e610782-336e-4df7-8dc4-a58a6defc347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fe15b9410f148d09dc3c5e5da9e8fc6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:31.984148 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e610782-336e-4df7-8dc4-a58a6defc347 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:32.031440 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7e610782-336e-4df7-8dc4-a58a6defc347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fe15b9410f148d09dc3c5e5da9e8fc6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:32.044193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7e610782-336e-4df7-8dc4-a58a6defc347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fe15b9410f148d09dc3c5e5da9e8fc6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:32.045052 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7e610782-336e-4df7-8dc4-a58a6defc347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fe15b9410f148d09dc3c5e5da9e8fc6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:32.045712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7e610782-336e-4df7-8dc4-a58a6defc347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fe15b9410f148d09dc3c5e5da9e8fc6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:32.052655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7e610782-336e-4df7-8dc4-a58a6defc347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fe15b9410f148d09dc3c5e5da9e8fc6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:32.066617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1287/2614] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:31 2022] DELETE /identity/v3/users/2fe15b9410f148d09dc3c5e5da9e8fc6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:32.071066 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4370e-d1f8-4cb2-b883-7605ddbc8246 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:32.071510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4370e-d1f8-4cb2-b883-7605ddbc8246 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:32.071676 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4370e-d1f8-4cb2-b883-7605ddbc8246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:32.089958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-cdf4370e-d1f8-4cb2-b883-7605ddbc8246 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:32.091154 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cdf4370e-d1f8-4cb2-b883-7605ddbc8246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:32.130932 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:32.150125 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1328/2615] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:32.159784 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e610782-336e-4df7-8dc4-a58a6defc347 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:32.161253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7e610782-336e-4df7-8dc4-a58a6defc347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:32.175676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:32.220697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e610782-336e-4df7-8dc4-a58a6defc347 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:32.224423 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f24e4a-97c2-4da3-a48e-82d932b84f13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:32.225072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f24e4a-97c2-4da3-a48e-82d932b84f13 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:32.225389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f24e4a-97c2-4da3-a48e-82d932b84f13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:32.225668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f24e4a-97c2-4da3-a48e-82d932b84f13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:32.235394 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f24e4a-97c2-4da3-a48e-82d932b84f13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:32.236817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f24e4a-97c2-4da3-a48e-82d932b84f13 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:32.247365 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1288/2616] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:32.516381 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4370e-d1f8-4cb2-b883-7605ddbc8246 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:32.518334 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cdf4370e-d1f8-4cb2-b883-7605ddbc8246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:32.536909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4370e-d1f8-4cb2-b883-7605ddbc8246 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:32.542144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199c1fa-c09d-4140-ac6e-97538a4309d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:32.543080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c1fa-c09d-4140-ac6e-97538a4309d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:32.544565 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c1fa-c09d-4140-ac6e-97538a4309d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:32.544959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199c1fa-c09d-4140-ac6e-97538a4309d2 admin admin] PATH_INFO: `/v3/projects/237cfd9d3875450f85e59aa266314621` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:32.546298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199c1fa-c09d-4140-ac6e-97538a4309d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=237cfd9d3875450f85e59aa266314621)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:32.548726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199c1fa-c09d-4140-ac6e-97538a4309d2 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:32.609017 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8199c1fa-c09d-4140-ac6e-97538a4309d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '237cfd9d3875450f85e59aa266314621', 'request_id': 'req-8199c1fa-c09d-4140-ac6e-97538a4309d2'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:32.614656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8199c1fa-c09d-4140-ac6e-97538a4309d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '237cfd9d3875450f85e59aa266314621', 'request_id': 'req-8199c1fa-c09d-4140-ac6e-97538a4309d2'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:32.624961 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8199c1fa-c09d-4140-ac6e-97538a4309d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '237cfd9d3875450f85e59aa266314621', 'request_id': 'req-8199c1fa-c09d-4140-ac6e-97538a4309d2'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:32.643684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8199c1fa-c09d-4140-ac6e-97538a4309d2 admin admin] The token cache is being invalidate because project 237cfd9d3875450f85e59aa266314621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:32.644188 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8199c1fa-c09d-4140-ac6e-97538a4309d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:32.645082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-8199c1fa-c09d-4140-ac6e-97538a4309d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:32.647548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1329/2617] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:32 2022] DELETE /identity/v3/projects/237cfd9d3875450f85e59aa266314621 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:32.960528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f24e4a-97c2-4da3-a48e-82d932b84f13 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:32.962013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-43f24e4a-97c2-4da3-a48e-82d932b84f13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:32.975295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f24e4a-97c2-4da3-a48e-82d932b84f13 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:33.034815 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b1f3d-d700-4a68-8553-8321e2078efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:33.035703 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b1f3d-d700-4a68-8553-8321e2078efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:33.035986 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b1f3d-d700-4a68-8553-8321e2078efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:33.036267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b1f3d-d700-4a68-8553-8321e2078efc admin admin] PATH_INFO: `/v3/users/2884f1b5b53845e5b11ceb6ce4e5ff47` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:33.037578 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b1f3d-d700-4a68-8553-8321e2078efc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2884f1b5b53845e5b11ceb6ce4e5ff47)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:33.039504 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b1f3d-d700-4a68-8553-8321e2078efc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:33.087296 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-147b1f3d-d700-4a68-8553-8321e2078efc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2884f1b5b53845e5b11ceb6ce4e5ff47'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:33.100569 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-147b1f3d-d700-4a68-8553-8321e2078efc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2884f1b5b53845e5b11ceb6ce4e5ff47'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:33.101271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-147b1f3d-d700-4a68-8553-8321e2078efc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2884f1b5b53845e5b11ceb6ce4e5ff47'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:33.101863 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-147b1f3d-d700-4a68-8553-8321e2078efc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2884f1b5b53845e5b11ceb6ce4e5ff47'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:33.109700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-147b1f3d-d700-4a68-8553-8321e2078efc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2884f1b5b53845e5b11ceb6ce4e5ff47'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:33.123330 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1289/2618] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:32 2022] DELETE /identity/v3/users/2884f1b5b53845e5b11ceb6ce4e5ff47 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:33.128531 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e08d3-c87f-4f00-9776-d22feef87080 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:33.129023 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e08d3-c87f-4f00-9776-d22feef87080 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:33.129121 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e08d3-c87f-4f00-9776-d22feef87080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:33.147138 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-9a8e08d3-c87f-4f00-9776-d22feef87080 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:33.148324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9a8e08d3-c87f-4f00-9776-d22feef87080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:33.211743 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1330/2619] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:33.221622 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b1f3d-d700-4a68-8553-8321e2078efc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:33.223043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-147b1f3d-d700-4a68-8553-8321e2078efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:33.259449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:33.283210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b1f3d-d700-4a68-8553-8321e2078efc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:33.286830 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e07fb-2c98-4bbe-abd5-3ff885b2c415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:33.287474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e07fb-2c98-4bbe-abd5-3ff885b2c415 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:33.287757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e07fb-2c98-4bbe-abd5-3ff885b2c415 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:33.288032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e07fb-2c98-4bbe-abd5-3ff885b2c415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:33.296991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e07fb-2c98-4bbe-abd5-3ff885b2c415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:33.298449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e07fb-2c98-4bbe-abd5-3ff885b2c415 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:33.307812 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1290/2620] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:33.575950 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e08d3-c87f-4f00-9776-d22feef87080 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:33.577403 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9a8e08d3-c87f-4f00-9776-d22feef87080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:33.590337 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e08d3-c87f-4f00-9776-d22feef87080 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:33.595242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:33.596621 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:33.596902 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:33.597387 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265 admin admin] PATH_INFO: `/v3/projects/c8cf22ccdb4644bfb347b3ef48adb4c8` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:33.598699 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8cf22ccdb4644bfb347b3ef48adb4c8)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:33.600982 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:33.648335 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8cf22ccdb4644bfb347b3ef48adb4c8', 'request_id': 'req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:33.653873 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8cf22ccdb4644bfb347b3ef48adb4c8', 'request_id': 'req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:33.659837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8cf22ccdb4644bfb347b3ef48adb4c8', 'request_id': 'req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:33.661497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:33.674051 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265 admin admin] The token cache is being invalidate because project c8cf22ccdb4644bfb347b3ef48adb4c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:33.674337 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:33.674963 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:33.676483 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1331/2621] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:33 2022] DELETE /identity/v3/projects/c8cf22ccdb4644bfb347b3ef48adb4c8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:33.680482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e07fb-2c98-4bbe-abd5-3ff885b2c415 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:33.690924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e07fb-2c98-4bbe-abd5-3ff885b2c415 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:33.749714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2b85a-be1b-449d-854c-6f0453c8a155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:33.750506 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2b85a-be1b-449d-854c-6f0453c8a155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:33.750782 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2b85a-be1b-449d-854c-6f0453c8a155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:33.751054 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2b85a-be1b-449d-854c-6f0453c8a155 admin admin] PATH_INFO: `/v3/users/8265eb8f430c4a12b15d9ccf7e9eb48b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:33.752224 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2b85a-be1b-449d-854c-6f0453c8a155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8265eb8f430c4a12b15d9ccf7e9eb48b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:33.754186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2b85a-be1b-449d-854c-6f0453c8a155 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:33.801167 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-92b2b85a-be1b-449d-854c-6f0453c8a155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8265eb8f430c4a12b15d9ccf7e9eb48b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:33.813304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-92b2b85a-be1b-449d-854c-6f0453c8a155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8265eb8f430c4a12b15d9ccf7e9eb48b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:33.813959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-92b2b85a-be1b-449d-854c-6f0453c8a155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8265eb8f430c4a12b15d9ccf7e9eb48b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:33.814557 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-92b2b85a-be1b-449d-854c-6f0453c8a155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8265eb8f430c4a12b15d9ccf7e9eb48b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:33.821466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-92b2b85a-be1b-449d-854c-6f0453c8a155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8265eb8f430c4a12b15d9ccf7e9eb48b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:33.834528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1291/2622] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:33 2022] DELETE /identity/v3/users/8265eb8f430c4a12b15d9ccf7e9eb48b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:34.131305 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:34.132912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:34.147195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc0935-1312-40f8-aec8-3f8a5e4b2265 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:34.229488 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cefa186-c5b1-41b2-bd81-04642d60a103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:34.230400 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cefa186-c5b1-41b2-bd81-04642d60a103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:34.231859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cefa186-c5b1-41b2-bd81-04642d60a103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:34.232161 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cefa186-c5b1-41b2-bd81-04642d60a103 admin admin] PATH_INFO: `/v3/projects/3f951f1d55f243d385cf37410a704713` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:34.233504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cefa186-c5b1-41b2-bd81-04642d60a103 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f951f1d55f243d385cf37410a704713)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:34.235417 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cefa186-c5b1-41b2-bd81-04642d60a103 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:34.295172 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7cefa186-c5b1-41b2-bd81-04642d60a103 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f951f1d55f243d385cf37410a704713', 'request_id': 'req-7cefa186-c5b1-41b2-bd81-04642d60a103'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:34.300617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7cefa186-c5b1-41b2-bd81-04642d60a103 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f951f1d55f243d385cf37410a704713', 'request_id': 'req-7cefa186-c5b1-41b2-bd81-04642d60a103'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:34.307043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7cefa186-c5b1-41b2-bd81-04642d60a103 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f951f1d55f243d385cf37410a704713', 'request_id': 'req-7cefa186-c5b1-41b2-bd81-04642d60a103'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:34.321716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7cefa186-c5b1-41b2-bd81-04642d60a103 admin admin] The token cache is being invalidate because project 3f951f1d55f243d385cf37410a704713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:34.322010 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7cefa186-c5b1-41b2-bd81-04642d60a103 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:34.322639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-7cefa186-c5b1-41b2-bd81-04642d60a103 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:34.324913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1332/2623] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:34 2022] DELETE /identity/v3/projects/3f951f1d55f243d385cf37410a704713 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:34.330089 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089373e0-bf2e-47ef-8b25-01a54b118a4a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:34.330537 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089373e0-bf2e-47ef-8b25-01a54b118a4a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:34.330650 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089373e0-bf2e-47ef-8b25-01a54b118a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:34.348579 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-089373e0-bf2e-47ef-8b25-01a54b118a4a None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:34.349822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-089373e0-bf2e-47ef-8b25-01a54b118a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:34.399115 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:34.412629 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1292/2624] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:34.422061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17805567-2996-455b-a97e-5874e78e3e80 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:34.422369 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17805567-2996-455b-a97e-5874e78e3e80 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:34.422655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17805567-2996-455b-a97e-5874e78e3e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:34.441530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-17805567-2996-455b-a97e-5874e78e3e80 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:34.442738 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-17805567-2996-455b-a97e-5874e78e3e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:34.450688 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1333/2625] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:34.454943 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089373e0-bf2e-47ef-8b25-01a54b118a4a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:34.464715 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089373e0-bf2e-47ef-8b25-01a54b118a4a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:34.469615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa73051-58ac-4981-aa0d-9a093b9eed51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:34.470279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa73051-58ac-4981-aa0d-9a093b9eed51 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:34.470565 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa73051-58ac-4981-aa0d-9a093b9eed51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:34.470845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa73051-58ac-4981-aa0d-9a093b9eed51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:34.471577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa73051-58ac-4981-aa0d-9a093b9eed51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:37:34.471890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa73051-58ac-4981-aa0d-9a093b9eed51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:34.473530 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa73051-58ac-4981-aa0d-9a093b9eed51 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:34.485787 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1293/2626] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:37:34.489863 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17805567-2996-455b-a97e-5874e78e3e80 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:34.499701 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17805567-2996-455b-a97e-5874e78e3e80 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:34.504345 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66e34a-747b-4d41-9939-2a886a30af82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:34.504991 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e34a-747b-4d41-9939-2a886a30af82 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:34.505322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e34a-747b-4d41-9939-2a886a30af82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:34.505720 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e34a-747b-4d41-9939-2a886a30af82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:34.506420 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66e34a-747b-4d41-9939-2a886a30af82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:34.508326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66e34a-747b-4d41-9939-2a886a30af82 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:34.533130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1334/2627] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:34 2022] POST /identity/v3/projects => generated 409 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:34.537241 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa73051-58ac-4981-aa0d-9a093b9eed51 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:34.547852 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa73051-58ac-4981-aa0d-9a093b9eed51 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:34.603071 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776891c-e9b1-4d68-87cc-c0fa61cbc72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:34.603741 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776891c-e9b1-4d68-87cc-c0fa61cbc72d admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:34.604073 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776891c-e9b1-4d68-87cc-c0fa61cbc72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:34.604353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776891c-e9b1-4d68-87cc-c0fa61cbc72d admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:34.605053 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776891c-e9b1-4d68-87cc-c0fa61cbc72d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:34.607479 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776891c-e9b1-4d68-87cc-c0fa61cbc72d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:34.627733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1294/2628] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:34 2022] POST /identity/v3/users => generated 305 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:37:34.631952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66e34a-747b-4d41-9939-2a886a30af82 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:34.642087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66e34a-747b-4d41-9939-2a886a30af82 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:34.646824 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d5b57-e058-4fc7-9079-45511817ac10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:34.647477 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d5b57-e058-4fc7-9079-45511817ac10 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:34.647755 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d5b57-e058-4fc7-9079-45511817ac10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:34.648035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d5b57-e058-4fc7-9079-45511817ac10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:34.648759 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d5b57-e058-4fc7-9079-45511817ac10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:34.650383 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d5b57-e058-4fc7-9079-45511817ac10 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:34.660386 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1335/2629] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:34.664109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776891c-e9b1-4d68-87cc-c0fa61cbc72d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:34.674704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776891c-e9b1-4d68-87cc-c0fa61cbc72d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:34.679276 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398b40f-9ce6-4689-942a-ba1da11e4dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:34.679947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398b40f-9ce6-4689-942a-ba1da11e4dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:34.680228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398b40f-9ce6-4689-942a-ba1da11e4dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:34.680504 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398b40f-9ce6-4689-942a-ba1da11e4dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:34.681265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398b40f-9ce6-4689-942a-ba1da11e4dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:34.682831 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398b40f-9ce6-4689-942a-ba1da11e4dc3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:34.693510 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1295/2630] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:34.697460 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d5b57-e058-4fc7-9079-45511817ac10 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:34.707324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d5b57-e058-4fc7-9079-45511817ac10 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:34.711971 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b63d8-f499-4be1-88cd-dca78366f652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:34.712829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b63d8-f499-4be1-88cd-dca78366f652 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:34.713266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b63d8-f499-4be1-88cd-dca78366f652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:34.713438 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b63d8-f499-4be1-88cd-dca78366f652 admin admin] PATH_INFO: `/v3/projects/d2d53219949d4a0e9182f8250d945acf/users/466bc7ac04c9442089ba74266b6f2b49/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:34.728699 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b63d8-f499-4be1-88cd-dca78366f652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2d53219949d4a0e9182f8250d945acf, user_id=466bc7ac04c9442089ba74266b6f2b49, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:34.732118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b63d8-f499-4be1-88cd-dca78366f652 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:34.748762 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1336/2631] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:34 2022] PUT /identity/v3/projects/d2d53219949d4a0e9182f8250d945acf/users/466bc7ac04c9442089ba74266b6f2b49/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:34.759143 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86c6a-eeb6-4b84-9acd-761dbf6657f9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:34.759435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86c6a-eeb6-4b84-9acd-761dbf6657f9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:34.759713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86c6a-eeb6-4b84-9acd-761dbf6657f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:34.787535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-77f86c6a-eeb6-4b84-9acd-761dbf6657f9 None None] MFA Rules not processed for user `466bc7ac04c9442089ba74266b6f2b49`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:34.788759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-77f86c6a-eeb6-4b84-9acd-761dbf6657f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:34.853527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1296/2632] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:34 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:34.858605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b63d8-f499-4be1-88cd-dca78366f652 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:34.877404 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:34.921760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b63d8-f499-4be1-88cd-dca78366f652 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:34.926400 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694c2df-1bdc-4e52-a2da-98b0e2089229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:34.927053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694c2df-1bdc-4e52-a2da-98b0e2089229 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:34.927338 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694c2df-1bdc-4e52-a2da-98b0e2089229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:34.927615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694c2df-1bdc-4e52-a2da-98b0e2089229 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:34.928311 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3694c2df-1bdc-4e52-a2da-98b0e2089229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:34.930254 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3694c2df-1bdc-4e52-a2da-98b0e2089229 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:34.954770 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1337/2633] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:34 2022] POST /identity/v3/projects => generated 407 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:34.958821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f86c6a-eeb6-4b84-9acd-761dbf6657f9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:34.970148 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f86c6a-eeb6-4b84-9acd-761dbf6657f9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:35.017222 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:35.029171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d21d3-6b7d-4461-88ff-811478ab1224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:35.030110 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d21d3-6b7d-4461-88ff-811478ab1224 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:35.030395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d21d3-6b7d-4461-88ff-811478ab1224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:35.030672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d21d3-6b7d-4461-88ff-811478ab1224 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:35.031359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d21d3-6b7d-4461-88ff-811478ab1224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:35.033324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d21d3-6b7d-4461-88ff-811478ab1224 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:35.053041 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1297/2634] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:34 2022] POST /identity/v3/users => generated 310 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:35.057065 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694c2df-1bdc-4e52-a2da-98b0e2089229 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:35.067277 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694c2df-1bdc-4e52-a2da-98b0e2089229 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:35.071848 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7af7a-0724-4e21-b3b4-f987a413e195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:35.072497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7af7a-0724-4e21-b3b4-f987a413e195 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:35.072783 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7af7a-0724-4e21-b3b4-f987a413e195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:35.073049 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7af7a-0724-4e21-b3b4-f987a413e195 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:35.073883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7af7a-0724-4e21-b3b4-f987a413e195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:35.076353 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7af7a-0724-4e21-b3b4-f987a413e195 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:35.086921 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1338/2635] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:35.091034 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d21d3-6b7d-4461-88ff-811478ab1224 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:35.101704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d21d3-6b7d-4461-88ff-811478ab1224 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:35.106359 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86171d-2554-473d-bab3-109bbe445c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:35.107174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86171d-2554-473d-bab3-109bbe445c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:35.107548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86171d-2554-473d-bab3-109bbe445c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:35.107829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86171d-2554-473d-bab3-109bbe445c43 admin admin] PATH_INFO: `/v3/projects/ffb69dcee8504624bbf6ae64efaac0ae/users/b8b13900db2b4837b039db53ff8f8918/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:35.122271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86171d-2554-473d-bab3-109bbe445c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb69dcee8504624bbf6ae64efaac0ae, user_id=b8b13900db2b4837b039db53ff8f8918, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:35.125251 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86171d-2554-473d-bab3-109bbe445c43 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:35.138745 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1298/2636] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:35 2022] PUT /identity/v3/projects/ffb69dcee8504624bbf6ae64efaac0ae/users/b8b13900db2b4837b039db53ff8f8918/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:35.143265 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7af7a-0724-4e21-b3b4-f987a413e195 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:35.153248 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7af7a-0724-4e21-b3b4-f987a413e195 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:35.209277 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cb1e1-1f9f-4e1d-9a16-d49847cb6bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:35.210072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cb1e1-1f9f-4e1d-9a16-d49847cb6bdb admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:35.210357 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cb1e1-1f9f-4e1d-9a16-d49847cb6bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:35.210656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cb1e1-1f9f-4e1d-9a16-d49847cb6bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:35.211374 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cb1e1-1f9f-4e1d-9a16-d49847cb6bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:35.212964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cb1e1-1f9f-4e1d-9a16-d49847cb6bdb admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:35.223398 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1339/2637] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:37:35.227746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86171d-2554-473d-bab3-109bbe445c43 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:35.239443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86171d-2554-473d-bab3-109bbe445c43 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:35.244556 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efafaef-7aa7-4b73-835b-bd82d05cedf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:35.245508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efafaef-7aa7-4b73-835b-bd82d05cedf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:35.245794 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efafaef-7aa7-4b73-835b-bd82d05cedf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:35.246070 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efafaef-7aa7-4b73-835b-bd82d05cedf8 admin admin] PATH_INFO: `/v3/projects/ffb69dcee8504624bbf6ae64efaac0ae/users/b8b13900db2b4837b039db53ff8f8918/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:35.253039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efafaef-7aa7-4b73-835b-bd82d05cedf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb69dcee8504624bbf6ae64efaac0ae, user_id=b8b13900db2b4837b039db53ff8f8918, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:35.253039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efafaef-7aa7-4b73-835b-bd82d05cedf8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:35.262237 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1299/2638] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:35 2022] PUT /identity/v3/projects/ffb69dcee8504624bbf6ae64efaac0ae/users/b8b13900db2b4837b039db53ff8f8918/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:35.275141 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec49633-29b6-468d-84f5-1d55d123e3c7 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:35.275434 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec49633-29b6-468d-84f5-1d55d123e3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:35.275719 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec49633-29b6-468d-84f5-1d55d123e3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:35.302258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-0ec49633-29b6-468d-84f5-1d55d123e3c7 None None] MFA Rules not processed for user `b8b13900db2b4837b039db53ff8f8918`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:35.303454 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0ec49633-29b6-468d-84f5-1d55d123e3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:35.370558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1340/2639] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:35 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:35.381687 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efafaef-7aa7-4b73-835b-bd82d05cedf8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:35.383432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4efafaef-7aa7-4b73-835b-bd82d05cedf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:35.441754 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efafaef-7aa7-4b73-835b-bd82d05cedf8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:35.445385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67499ca0-44b0-419f-9a6b-0f183135b55b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:35.446030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67499ca0-44b0-419f-9a6b-0f183135b55b service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:35.446315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67499ca0-44b0-419f-9a6b-0f183135b55b service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:35.446594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67499ca0-44b0-419f-9a6b-0f183135b55b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:35.456102 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67499ca0-44b0-419f-9a6b-0f183135b55b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:35.457577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67499ca0-44b0-419f-9a6b-0f183135b55b service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:35.466908 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1300/2640] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:35 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:35.610460 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec49633-29b6-468d-84f5-1d55d123e3c7 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:35.620003 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec49633-29b6-468d-84f5-1d55d123e3c7 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:35.624213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171d040-ef53-4a64-817b-58c436c72d43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:35.624791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171d040-ef53-4a64-817b-58c436c72d43 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:35.625076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171d040-ef53-4a64-817b-58c436c72d43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:35.625385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171d040-ef53-4a64-817b-58c436c72d43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:35.634589 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:35.682066 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171d040-ef53-4a64-817b-58c436c72d43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:35.683504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171d040-ef53-4a64-817b-58c436c72d43 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:35.692996 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1341/2641] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:35 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:44.117941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67499ca0-44b0-419f-9a6b-0f183135b55b service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:44.211247 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67499ca0-44b0-419f-9a6b-0f183135b55b service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:44.227307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcbe02-4359-4f78-b80d-9fa5940d8a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:44.228854 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcbe02-4359-4f78-b80d-9fa5940d8a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:44.229689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcbe02-4359-4f78-b80d-9fa5940d8a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:44.230008 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcbe02-4359-4f78-b80d-9fa5940d8a21 admin admin] PATH_INFO: `/v3/users/466bc7ac04c9442089ba74266b6f2b49` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:44.232593 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fcbe02-4359-4f78-b80d-9fa5940d8a21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=466bc7ac04c9442089ba74266b6f2b49)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:44.236819 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fcbe02-4359-4f78-b80d-9fa5940d8a21 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:44.317423 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-82fcbe02-4359-4f78-b80d-9fa5940d8a21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '466bc7ac04c9442089ba74266b6f2b49'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:44.323725 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:44.344925 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-82fcbe02-4359-4f78-b80d-9fa5940d8a21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '466bc7ac04c9442089ba74266b6f2b49'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:44.345734 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-82fcbe02-4359-4f78-b80d-9fa5940d8a21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '466bc7ac04c9442089ba74266b6f2b49'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:44.346335 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-82fcbe02-4359-4f78-b80d-9fa5940d8a21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '466bc7ac04c9442089ba74266b6f2b49'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:44.355725 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-82fcbe02-4359-4f78-b80d-9fa5940d8a21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '466bc7ac04c9442089ba74266b6f2b49'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:44.377040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1301/2642] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:44 2022] DELETE /identity/v3/users/466bc7ac04c9442089ba74266b6f2b49 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:44.382072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e91a5-647e-4ff0-87b5-c6adae65300d None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:44.382675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e91a5-647e-4ff0-87b5-c6adae65300d None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:44.382675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e91a5-647e-4ff0-87b5-c6adae65300d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:44.402940 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-b73e91a5-647e-4ff0-87b5-c6adae65300d None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:44.404950 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b73e91a5-647e-4ff0-87b5-c6adae65300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:44.471955 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1342/2643] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:44.481740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcbe02-4359-4f78-b80d-9fa5940d8a21 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:44.483247 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-82fcbe02-4359-4f78-b80d-9fa5940d8a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:44.546757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcbe02-4359-4f78-b80d-9fa5940d8a21 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:44.550517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac0561-4ad2-4422-a83e-13835aa60b8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:44.551170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac0561-4ad2-4422-a83e-13835aa60b8d service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:44.551457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac0561-4ad2-4422-a83e-13835aa60b8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:44.551737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac0561-4ad2-4422-a83e-13835aa60b8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:44.562131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac0561-4ad2-4422-a83e-13835aa60b8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:44.563576 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac0561-4ad2-4422-a83e-13835aa60b8d service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:44.574153 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1302/2644] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:44.862206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e91a5-647e-4ff0-87b5-c6adae65300d None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:44.863900 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b73e91a5-647e-4ff0-87b5-c6adae65300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:44.877331 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e91a5-647e-4ff0-87b5-c6adae65300d None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:44.882165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:44.883036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:44.883460 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:44.883746 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b admin admin] PATH_INFO: `/v3/projects/d2d53219949d4a0e9182f8250d945acf` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:44.884970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2d53219949d4a0e9182f8250d945acf)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:44.887082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:44.940401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2d53219949d4a0e9182f8250d945acf', 'request_id': 'req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:44.946598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2d53219949d4a0e9182f8250d945acf', 'request_id': 'req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:44.953742 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2d53219949d4a0e9182f8250d945acf', 'request_id': 'req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:44.967521 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b admin admin] The token cache is being invalidate because project d2d53219949d4a0e9182f8250d945acf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:44.967816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:44.968430 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:44.970065 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1343/2645] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:44 2022] DELETE /identity/v3/projects/d2d53219949d4a0e9182f8250d945acf => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:44.973942 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac0561-4ad2-4422-a83e-13835aa60b8d service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:44.984121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac0561-4ad2-4422-a83e-13835aa60b8d service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:45.042718 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3400b-495a-411c-b82b-e29b17ab5ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:45.043945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3400b-495a-411c-b82b-e29b17ab5ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:45.045027 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3400b-495a-411c-b82b-e29b17ab5ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:45.045431 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3400b-495a-411c-b82b-e29b17ab5ce8 admin admin] PATH_INFO: `/v3/users/b8b13900db2b4837b039db53ff8f8918` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:45.047082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3400b-495a-411c-b82b-e29b17ab5ce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8b13900db2b4837b039db53ff8f8918)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:45.049168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3400b-495a-411c-b82b-e29b17ab5ce8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:45.099796 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-29d3400b-495a-411c-b82b-e29b17ab5ce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b13900db2b4837b039db53ff8f8918'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:45.113363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-29d3400b-495a-411c-b82b-e29b17ab5ce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b13900db2b4837b039db53ff8f8918'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:45.114199 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-29d3400b-495a-411c-b82b-e29b17ab5ce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b13900db2b4837b039db53ff8f8918'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:45.114740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-29d3400b-495a-411c-b82b-e29b17ab5ce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b13900db2b4837b039db53ff8f8918'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:45.125200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:45.132060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-29d3400b-495a-411c-b82b-e29b17ab5ce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b13900db2b4837b039db53ff8f8918'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:45.147138 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:45.148680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:45.153907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1303/2646] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:44 2022] DELETE /identity/v3/users/b8b13900db2b4837b039db53ff8f8918 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:45.168243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9db09-6afb-4760-b1bb-7ce00a0c744b admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:45.222749 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:45.266422 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6d217-0a82-4583-b249-196ea170c8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:45.267853 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6d217-0a82-4583-b249-196ea170c8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:45.268167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6d217-0a82-4583-b249-196ea170c8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:45.268501 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6d217-0a82-4583-b249-196ea170c8f4 admin admin] PATH_INFO: `/v3/users/e07e9457e5f94eae9381cd236b049504` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:45.269902 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6d217-0a82-4583-b249-196ea170c8f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e07e9457e5f94eae9381cd236b049504)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:45.271912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6d217-0a82-4583-b249-196ea170c8f4 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:45.332147 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-bce6d217-0a82-4583-b249-196ea170c8f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e07e9457e5f94eae9381cd236b049504'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:45.349705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-bce6d217-0a82-4583-b249-196ea170c8f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e07e9457e5f94eae9381cd236b049504'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:45.350451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-bce6d217-0a82-4583-b249-196ea170c8f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e07e9457e5f94eae9381cd236b049504'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:45.359669 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-bce6d217-0a82-4583-b249-196ea170c8f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e07e9457e5f94eae9381cd236b049504'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:45.360443 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-bce6d217-0a82-4583-b249-196ea170c8f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e07e9457e5f94eae9381cd236b049504'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:45.377946 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1344/2647] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:45 2022] DELETE /identity/v3/users/e07e9457e5f94eae9381cd236b049504 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:45.383126 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148e855-4fa2-4cc6-ad55-63360fd6c6d7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:45.383385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148e855-4fa2-4cc6-ad55-63360fd6c6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:45.383726 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148e855-4fa2-4cc6-ad55-63360fd6c6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:45.405595 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b148e855-4fa2-4cc6-ad55-63360fd6c6d7 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:45.407676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b148e855-4fa2-4cc6-ad55-63360fd6c6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:45.516833 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1304/2648] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:45.531494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6d217-0a82-4583-b249-196ea170c8f4 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:45.534157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-bce6d217-0a82-4583-b249-196ea170c8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:45.602176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6d217-0a82-4583-b249-196ea170c8f4 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:45.610280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73c82f-181b-42b4-9d10-e2f126195e67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:45.610999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73c82f-181b-42b4-9d10-e2f126195e67 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:45.611380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73c82f-181b-42b4-9d10-e2f126195e67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:45.611728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73c82f-181b-42b4-9d10-e2f126195e67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:45.622690 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73c82f-181b-42b4-9d10-e2f126195e67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:45.624579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73c82f-181b-42b4-9d10-e2f126195e67 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:45.625625 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148e855-4fa2-4cc6-ad55-63360fd6c6d7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:45.627141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b148e855-4fa2-4cc6-ad55-63360fd6c6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:45.635712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1345/2649] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:45.644991 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148e855-4fa2-4cc6-ad55-63360fd6c6d7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:45.650161 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834efb08-8493-44fb-bbb6-25a10ef7f4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:45.651291 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834efb08-8493-44fb-bbb6-25a10ef7f4ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:45.651675 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834efb08-8493-44fb-bbb6-25a10ef7f4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:45.652059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834efb08-8493-44fb-bbb6-25a10ef7f4ae admin admin] PATH_INFO: `/v3/projects/ffb69dcee8504624bbf6ae64efaac0ae` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:45.654076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834efb08-8493-44fb-bbb6-25a10ef7f4ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffb69dcee8504624bbf6ae64efaac0ae)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:45.656393 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834efb08-8493-44fb-bbb6-25a10ef7f4ae admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:45.715339 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-834efb08-8493-44fb-bbb6-25a10ef7f4ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffb69dcee8504624bbf6ae64efaac0ae', 'request_id': 'req-834efb08-8493-44fb-bbb6-25a10ef7f4ae'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:45.726058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-834efb08-8493-44fb-bbb6-25a10ef7f4ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffb69dcee8504624bbf6ae64efaac0ae', 'request_id': 'req-834efb08-8493-44fb-bbb6-25a10ef7f4ae'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:45.734719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-834efb08-8493-44fb-bbb6-25a10ef7f4ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffb69dcee8504624bbf6ae64efaac0ae', 'request_id': 'req-834efb08-8493-44fb-bbb6-25a10ef7f4ae'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:45.747467 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-834efb08-8493-44fb-bbb6-25a10ef7f4ae admin admin] The token cache is being invalidate because project ffb69dcee8504624bbf6ae64efaac0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:45.747467 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-834efb08-8493-44fb-bbb6-25a10ef7f4ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:45.748405 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-834efb08-8493-44fb-bbb6-25a10ef7f4ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:45.750515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1305/2650] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:45 2022] DELETE /identity/v3/projects/ffb69dcee8504624bbf6ae64efaac0ae => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:45.755840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6443614-11fc-419c-81e2-e717023cdc9f None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:45.756137 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6443614-11fc-419c-81e2-e717023cdc9f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:45.756418 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6443614-11fc-419c-81e2-e717023cdc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:45.777099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-a6443614-11fc-419c-81e2-e717023cdc9f None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:45.779338 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a6443614-11fc-419c-81e2-e717023cdc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:45.835624 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:45.855460 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1346/2651] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:45.864480 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b917a-a03b-4327-9c14-97e3bc5fff92 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:45.864842 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b917a-a03b-4327-9c14-97e3bc5fff92 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:45.865212 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b917a-a03b-4327-9c14-97e3bc5fff92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:45.884849 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-224b917a-a03b-4327-9c14-97e3bc5fff92 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:45.886255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-224b917a-a03b-4327-9c14-97e3bc5fff92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:45.896920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1306/2652] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:45.901197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6443614-11fc-419c-81e2-e717023cdc9f None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:45.911671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6443614-11fc-419c-81e2-e717023cdc9f None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:45.916286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e50c12-567c-4d43-911e-417ce98da05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:45.916946 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e50c12-567c-4d43-911e-417ce98da05e admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:45.917259 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e50c12-567c-4d43-911e-417ce98da05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:45.917532 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e50c12-567c-4d43-911e-417ce98da05e admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:45.918275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e50c12-567c-4d43-911e-417ce98da05e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:37:45.918582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e50c12-567c-4d43-911e-417ce98da05e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:45.920228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e50c12-567c-4d43-911e-417ce98da05e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:45.932857 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1347/2653] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:37:45.937643 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b917a-a03b-4327-9c14-97e3bc5fff92 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:45.947437 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b917a-a03b-4327-9c14-97e3bc5fff92 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:45.953320 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74fee1-a955-435d-afa8-c5fea3f6feaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:45.954111 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74fee1-a955-435d-afa8-c5fea3f6feaf admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:45.954491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74fee1-a955-435d-afa8-c5fea3f6feaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:45.954859 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74fee1-a955-435d-afa8-c5fea3f6feaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:45.955662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74fee1-a955-435d-afa8-c5fea3f6feaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:45.958261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74fee1-a955-435d-afa8-c5fea3f6feaf admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:45.986757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1307/2654] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:45 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:45.991382 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e50c12-567c-4d43-911e-417ce98da05e admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:46.004665 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e50c12-567c-4d43-911e-417ce98da05e admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:46.072688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74fee1-a955-435d-afa8-c5fea3f6feaf admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:46.075131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5f74fee1-a955-435d-afa8-c5fea3f6feaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:46.076964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac224a1d-7e2d-49ef-b284-9dadeff02578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:46.077980 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac224a1d-7e2d-49ef-b284-9dadeff02578 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:46.078531 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac224a1d-7e2d-49ef-b284-9dadeff02578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:46.078911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac224a1d-7e2d-49ef-b284-9dadeff02578 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:46.079643 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac224a1d-7e2d-49ef-b284-9dadeff02578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:46.081602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac224a1d-7e2d-49ef-b284-9dadeff02578 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:46.094344 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74fee1-a955-435d-afa8-c5fea3f6feaf admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:46.104853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c973a0d-3048-4bd4-942d-0eff1db9d87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:46.118722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c973a0d-3048-4bd4-942d-0eff1db9d87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:46.118722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c973a0d-3048-4bd4-942d-0eff1db9d87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:46.118722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c973a0d-3048-4bd4-942d-0eff1db9d87c admin admin] PATH_INFO: `/v3/projects/1e5ade04ee1e46a3b56135bef158e1d1` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:46.118722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c973a0d-3048-4bd4-942d-0eff1db9d87c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e5ade04ee1e46a3b56135bef158e1d1)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:46.119140 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1348/2655] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:45 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:46.119238 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c973a0d-3048-4bd4-942d-0eff1db9d87c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:46.121440 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac224a1d-7e2d-49ef-b284-9dadeff02578 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:46.135695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac224a1d-7e2d-49ef-b284-9dadeff02578 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:46.141950 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501daf7-3f39-41d0-97de-c59c168bc566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:46.142948 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501daf7-3f39-41d0-97de-c59c168bc566 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:46.143342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501daf7-3f39-41d0-97de-c59c168bc566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:46.143831 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501daf7-3f39-41d0-97de-c59c168bc566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:46.144611 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501daf7-3f39-41d0-97de-c59c168bc566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:46.146429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501daf7-3f39-41d0-97de-c59c168bc566 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:46.158651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1349/2656] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:46.162602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501daf7-3f39-41d0-97de-c59c168bc566 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:46.164869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:46.174949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501daf7-3f39-41d0-97de-c59c168bc566 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:46.177487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7c973a0d-3048-4bd4-942d-0eff1db9d87c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e5ade04ee1e46a3b56135bef158e1d1', 'request_id': 'req-7c973a0d-3048-4bd4-942d-0eff1db9d87c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:46.186323 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7c973a0d-3048-4bd4-942d-0eff1db9d87c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e5ade04ee1e46a3b56135bef158e1d1', 'request_id': 'req-7c973a0d-3048-4bd4-942d-0eff1db9d87c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:46.194260 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7c973a0d-3048-4bd4-942d-0eff1db9d87c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e5ade04ee1e46a3b56135bef158e1d1', 'request_id': 'req-7c973a0d-3048-4bd4-942d-0eff1db9d87c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:46.208748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7c973a0d-3048-4bd4-942d-0eff1db9d87c admin admin] The token cache is being invalidate because project 1e5ade04ee1e46a3b56135bef158e1d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:46.209040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7c973a0d-3048-4bd4-942d-0eff1db9d87c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:46.209792 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7c973a0d-3048-4bd4-942d-0eff1db9d87c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:46.211364 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1308/2657] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:46 2022] DELETE /identity/v3/projects/1e5ade04ee1e46a3b56135bef158e1d1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:46.263818 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840e95c-ca28-4b0c-b2ce-9f63d86f71aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:46.264538 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840e95c-ca28-4b0c-b2ce-9f63d86f71aa admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:46.264821 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840e95c-ca28-4b0c-b2ce-9f63d86f71aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:46.265099 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840e95c-ca28-4b0c-b2ce-9f63d86f71aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:46.265889 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840e95c-ca28-4b0c-b2ce-9f63d86f71aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:46.267794 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840e95c-ca28-4b0c-b2ce-9f63d86f71aa admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:46.279537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1350/2658] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:37:46.283558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c973a0d-3048-4bd4-942d-0eff1db9d87c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:46.285056 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7c973a0d-3048-4bd4-942d-0eff1db9d87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:46.299333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c973a0d-3048-4bd4-942d-0eff1db9d87c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:46.304223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa30fb7-3244-4323-b6b0-099a82ed7f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:46.305308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa30fb7-3244-4323-b6b0-099a82ed7f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:46.305661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa30fb7-3244-4323-b6b0-099a82ed7f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:46.306002 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa30fb7-3244-4323-b6b0-099a82ed7f70 admin admin] PATH_INFO: `/v3/projects/c45f32fba7a14bf8aa83d330622c50a9/users/e182de23f1554abebac8f39771f23e12/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:46.323870 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa30fb7-3244-4323-b6b0-099a82ed7f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c45f32fba7a14bf8aa83d330622c50a9, user_id=e182de23f1554abebac8f39771f23e12, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:46.329189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa30fb7-3244-4323-b6b0-099a82ed7f70 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:46.346590 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1309/2659] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:46 2022] PUT /identity/v3/projects/c45f32fba7a14bf8aa83d330622c50a9/users/e182de23f1554abebac8f39771f23e12/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:46.359263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513e886-e5fb-4cf5-a4ee-1fcbbc754adb None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:46.359693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513e886-e5fb-4cf5-a4ee-1fcbbc754adb None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:46.360210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513e886-e5fb-4cf5-a4ee-1fcbbc754adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:46.395420 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-0513e886-e5fb-4cf5-a4ee-1fcbbc754adb None None] MFA Rules not processed for user `e182de23f1554abebac8f39771f23e12`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:46.396920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-0513e886-e5fb-4cf5-a4ee-1fcbbc754adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:46.458545 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:46.478678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1351/2660] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:46.483387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa30fb7-3244-4323-b6b0-099a82ed7f70 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:46.544048 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa30fb7-3244-4323-b6b0-099a82ed7f70 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:46.548671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c472cd-ce59-48a3-abf0-630ab42e08ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:46.549349 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c472cd-ce59-48a3-abf0-630ab42e08ce admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:46.549632 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c472cd-ce59-48a3-abf0-630ab42e08ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:46.549907 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c472cd-ce59-48a3-abf0-630ab42e08ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:46.550624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c472cd-ce59-48a3-abf0-630ab42e08ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:46.552514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c472cd-ce59-48a3-abf0-630ab42e08ce admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:46.578146 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1310/2661] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:46 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:46.582158 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513e886-e5fb-4cf5-a4ee-1fcbbc754adb None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:46.593359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513e886-e5fb-4cf5-a4ee-1fcbbc754adb None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:46.648036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d048d11-be65-4f39-8c36-8c80044142d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:46.648627 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d048d11-be65-4f39-8c36-8c80044142d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:46.648909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d048d11-be65-4f39-8c36-8c80044142d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:46.649212 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d048d11-be65-4f39-8c36-8c80044142d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:46.649896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d048d11-be65-4f39-8c36-8c80044142d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:46.651799 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d048d11-be65-4f39-8c36-8c80044142d2 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:46.671830 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1352/2662] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:46 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:46.675727 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c472cd-ce59-48a3-abf0-630ab42e08ce admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:46.686541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c472cd-ce59-48a3-abf0-630ab42e08ce admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:46.691077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094f5a4-f670-4a03-8a7b-a59fc5940e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:46.691731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094f5a4-f670-4a03-8a7b-a59fc5940e63 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:46.692015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094f5a4-f670-4a03-8a7b-a59fc5940e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:46.692297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094f5a4-f670-4a03-8a7b-a59fc5940e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:46.693022 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094f5a4-f670-4a03-8a7b-a59fc5940e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:46.694632 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094f5a4-f670-4a03-8a7b-a59fc5940e63 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:46.706013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1311/2663] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:46.709834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d048d11-be65-4f39-8c36-8c80044142d2 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:46.720008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d048d11-be65-4f39-8c36-8c80044142d2 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:46.724700 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0eabf-4195-4952-b3e6-e80e904f5415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:46.725385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0eabf-4195-4952-b3e6-e80e904f5415 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:46.725677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0eabf-4195-4952-b3e6-e80e904f5415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:46.725956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0eabf-4195-4952-b3e6-e80e904f5415 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:46.726696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0eabf-4195-4952-b3e6-e80e904f5415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:46.728290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0eabf-4195-4952-b3e6-e80e904f5415 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:46.738270 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1353/2664] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:46.742247 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094f5a4-f670-4a03-8a7b-a59fc5940e63 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:46.752767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094f5a4-f670-4a03-8a7b-a59fc5940e63 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:46.757418 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c481e-4771-4e4c-93a0-dd54dfc0d536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:46.758288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c481e-4771-4e4c-93a0-dd54dfc0d536 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:46.758569 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c481e-4771-4e4c-93a0-dd54dfc0d536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:46.758844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c481e-4771-4e4c-93a0-dd54dfc0d536 admin admin] PATH_INFO: `/v3/projects/0812cb6fe1e04c57a1e9341a484a0846/users/591d4cfdf6c54654ac9130594c67cb98/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:46.774863 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c481e-4771-4e4c-93a0-dd54dfc0d536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0812cb6fe1e04c57a1e9341a484a0846, user_id=591d4cfdf6c54654ac9130594c67cb98, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:46.777486 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c481e-4771-4e4c-93a0-dd54dfc0d536 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:46.791370 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1312/2665] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:46 2022] PUT /identity/v3/projects/0812cb6fe1e04c57a1e9341a484a0846/users/591d4cfdf6c54654ac9130594c67cb98/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:46.803068 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf2b81-b9f8-45aa-b8e3-6527b473498f None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:46.803362 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf2b81-b9f8-45aa-b8e3-6527b473498f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:46.803667 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf2b81-b9f8-45aa-b8e3-6527b473498f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:46.830718 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-7ccf2b81-b9f8-45aa-b8e3-6527b473498f None None] MFA Rules not processed for user `591d4cfdf6c54654ac9130594c67cb98`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:46.831943 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7ccf2b81-b9f8-45aa-b8e3-6527b473498f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:46.896499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1354/2666] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:46.901001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c481e-4771-4e4c-93a0-dd54dfc0d536 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:46.951146 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:46.963719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c481e-4771-4e4c-93a0-dd54dfc0d536 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:46.969260 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bafbb7b-d1ed-4e02-93f9-66fa58c9cbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:46.969943 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafbb7b-d1ed-4e02-93f9-66fa58c9cbd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:46.970226 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafbb7b-d1ed-4e02-93f9-66fa58c9cbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:46.970503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafbb7b-d1ed-4e02-93f9-66fa58c9cbd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:46.971183 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bafbb7b-d1ed-4e02-93f9-66fa58c9cbd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:46.973065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bafbb7b-d1ed-4e02-93f9-66fa58c9cbd7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:46.998007 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1313/2667] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:46 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:47.002045 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf2b81-b9f8-45aa-b8e3-6527b473498f None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:47.012861 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf2b81-b9f8-45aa-b8e3-6527b473498f None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:47.064085 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:47.070887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0be751-be96-40ea-957a-d8d3aa3561af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:47.071554 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0be751-be96-40ea-957a-d8d3aa3561af admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:47.071837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0be751-be96-40ea-957a-d8d3aa3561af admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:47.072113 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0be751-be96-40ea-957a-d8d3aa3561af admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:47.072831 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0be751-be96-40ea-957a-d8d3aa3561af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:47.075823 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0be751-be96-40ea-957a-d8d3aa3561af admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:47.096351 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1355/2668] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:47 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:37:47.100342 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bafbb7b-d1ed-4e02-93f9-66fa58c9cbd7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:47.110817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bafbb7b-d1ed-4e02-93f9-66fa58c9cbd7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:47.115456 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b434532-cf47-4aa9-8261-334467c5e6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:47.116110 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b434532-cf47-4aa9-8261-334467c5e6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:47.116396 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b434532-cf47-4aa9-8261-334467c5e6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:47.116672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b434532-cf47-4aa9-8261-334467c5e6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:47.117435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b434532-cf47-4aa9-8261-334467c5e6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:47.119013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b434532-cf47-4aa9-8261-334467c5e6b4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:47.130180 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1314/2669] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:47.134093 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0be751-be96-40ea-957a-d8d3aa3561af admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:47.144584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0be751-be96-40ea-957a-d8d3aa3561af admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:47.149039 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56366e0f-34db-4642-b735-c8c60c4f3e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:47.149928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56366e0f-34db-4642-b735-c8c60c4f3e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:47.150206 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56366e0f-34db-4642-b735-c8c60c4f3e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:47.150478 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56366e0f-34db-4642-b735-c8c60c4f3e7e admin admin] PATH_INFO: `/v3/projects/daf9d9082de9478a9a49676371a5a67f/users/a73d43f0815b480799e6d55d436cecc6/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:47.165845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56366e0f-34db-4642-b735-c8c60c4f3e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf9d9082de9478a9a49676371a5a67f, user_id=a73d43f0815b480799e6d55d436cecc6, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:47.168426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56366e0f-34db-4642-b735-c8c60c4f3e7e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:47.181380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1356/2670] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:47 2022] PUT /identity/v3/projects/daf9d9082de9478a9a49676371a5a67f/users/a73d43f0815b480799e6d55d436cecc6/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:47.185430 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b434532-cf47-4aa9-8261-334467c5e6b4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:47.195497 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b434532-cf47-4aa9-8261-334467c5e6b4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:47.252756 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163993b-85de-49f5-abb2-d2b4836ce8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:47.253474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163993b-85de-49f5-abb2-d2b4836ce8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:47.256192 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163993b-85de-49f5-abb2-d2b4836ce8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:47.256484 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163993b-85de-49f5-abb2-d2b4836ce8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:47.257279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163993b-85de-49f5-abb2-d2b4836ce8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:47.258881 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163993b-85de-49f5-abb2-d2b4836ce8f4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:47.272916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1315/2671] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:47.277534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56366e0f-34db-4642-b735-c8c60c4f3e7e admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:47.288639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56366e0f-34db-4642-b735-c8c60c4f3e7e admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:47.293221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede88b62-3318-4d33-807e-96c2f6f265f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:47.294133 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede88b62-3318-4d33-807e-96c2f6f265f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:47.294414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede88b62-3318-4d33-807e-96c2f6f265f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:47.294682 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede88b62-3318-4d33-807e-96c2f6f265f1 admin admin] PATH_INFO: `/v3/projects/daf9d9082de9478a9a49676371a5a67f/users/a73d43f0815b480799e6d55d436cecc6/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:47.296651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede88b62-3318-4d33-807e-96c2f6f265f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf9d9082de9478a9a49676371a5a67f, user_id=a73d43f0815b480799e6d55d436cecc6, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:47.299282 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede88b62-3318-4d33-807e-96c2f6f265f1 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:47.316942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1357/2672] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:47 2022] PUT /identity/v3/projects/daf9d9082de9478a9a49676371a5a67f/users/a73d43f0815b480799e6d55d436cecc6/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:47.327759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfbc76-7999-4091-9db5-036736815d9e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:47.328057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfbc76-7999-4091-9db5-036736815d9e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:47.328349 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfbc76-7999-4091-9db5-036736815d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:47.363937 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d0dfbc76-7999-4091-9db5-036736815d9e None None] MFA Rules not processed for user `a73d43f0815b480799e6d55d436cecc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:47.365360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d0dfbc76-7999-4091-9db5-036736815d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:47.442548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1316/2673] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:47.455910 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede88b62-3318-4d33-807e-96c2f6f265f1 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:47.457032 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ede88b62-3318-4d33-807e-96c2f6f265f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:47.514526 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede88b62-3318-4d33-807e-96c2f6f265f1 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:47.518452 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6664ec2-a03f-4255-90df-0fbd3cb51c37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:47.519115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6664ec2-a03f-4255-90df-0fbd3cb51c37 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:47.519401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6664ec2-a03f-4255-90df-0fbd3cb51c37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:47.519678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6664ec2-a03f-4255-90df-0fbd3cb51c37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:47.529664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6664ec2-a03f-4255-90df-0fbd3cb51c37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:47.531116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6664ec2-a03f-4255-90df-0fbd3cb51c37 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:47.541561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1358/2674] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:47.583134 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dfbc76-7999-4091-9db5-036736815d9e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:47.584561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d0dfbc76-7999-4091-9db5-036736815d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:47.598668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dfbc76-7999-4091-9db5-036736815d9e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:47.602336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534ca763-27af-4d9b-85ca-08af27827b80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:47.602986 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ca763-27af-4d9b-85ca-08af27827b80 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:47.603270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ca763-27af-4d9b-85ca-08af27827b80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:47.603547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ca763-27af-4d9b-85ca-08af27827b80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:47.650059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:47.663874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534ca763-27af-4d9b-85ca-08af27827b80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:47.665341 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534ca763-27af-4d9b-85ca-08af27827b80 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:47.674619 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1317/2675] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:48.560578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6664ec2-a03f-4255-90df-0fbd3cb51c37 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:48.562046 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d6664ec2-a03f-4255-90df-0fbd3cb51c37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:48.619122 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6664ec2-a03f-4255-90df-0fbd3cb51c37 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:48.623362 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f8d81-4667-4b45-8bf3-4f9d4c3a8d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:48.624010 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f8d81-4667-4b45-8bf3-4f9d4c3a8d11 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:48.624314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f8d81-4667-4b45-8bf3-4f9d4c3a8d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:48.624592 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f8d81-4667-4b45-8bf3-4f9d4c3a8d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:48.626609 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-354f8d81-4667-4b45-8bf3-4f9d4c3a8d11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:48.688834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f8d81-4667-4b45-8bf3-4f9d4c3a8d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:48.690314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f8d81-4667-4b45-8bf3-4f9d4c3a8d11 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:48.700107 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1359/2676] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:50.540390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534ca763-27af-4d9b-85ca-08af27827b80 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:50.555057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534ca763-27af-4d9b-85ca-08af27827b80 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:50.560089 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af309b0-5608-4310-a51d-2ff627b5c27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:50.561229 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af309b0-5608-4310-a51d-2ff627b5c27b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:50.561664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af309b0-5608-4310-a51d-2ff627b5c27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:50.562092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af309b0-5608-4310-a51d-2ff627b5c27b admin admin] PATH_INFO: `/v3/users/e182de23f1554abebac8f39771f23e12` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:50.563450 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af309b0-5608-4310-a51d-2ff627b5c27b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e182de23f1554abebac8f39771f23e12)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:50.567162 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af309b0-5608-4310-a51d-2ff627b5c27b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:50.631052 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1af309b0-5608-4310-a51d-2ff627b5c27b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e182de23f1554abebac8f39771f23e12'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:50.645725 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1af309b0-5608-4310-a51d-2ff627b5c27b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e182de23f1554abebac8f39771f23e12'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:50.646472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1af309b0-5608-4310-a51d-2ff627b5c27b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e182de23f1554abebac8f39771f23e12'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:50.647076 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1af309b0-5608-4310-a51d-2ff627b5c27b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e182de23f1554abebac8f39771f23e12'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:50.657043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1af309b0-5608-4310-a51d-2ff627b5c27b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e182de23f1554abebac8f39771f23e12'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:50.673122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1318/2677] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:50 2022] DELETE /identity/v3/users/e182de23f1554abebac8f39771f23e12 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:50.677771 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12693860-b5e8-42f2-80bc-2b2a26e50bb9 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:50.678288 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12693860-b5e8-42f2-80bc-2b2a26e50bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:50.678399 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12693860-b5e8-42f2-80bc-2b2a26e50bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:50.688009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:50.702558 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-12693860-b5e8-42f2-80bc-2b2a26e50bb9 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:50.703778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-12693860-b5e8-42f2-80bc-2b2a26e50bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:50.777679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1360/2678] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:50.784964 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af309b0-5608-4310-a51d-2ff627b5c27b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:50.786433 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1af309b0-5608-4310-a51d-2ff627b5c27b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:50.859079 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af309b0-5608-4310-a51d-2ff627b5c27b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:50.862770 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66853637-7c84-4e12-a624-3313fe7acae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:50.863426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66853637-7c84-4e12-a624-3313fe7acae7 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:50.863712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66853637-7c84-4e12-a624-3313fe7acae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:50.863992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66853637-7c84-4e12-a624-3313fe7acae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:50.873886 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66853637-7c84-4e12-a624-3313fe7acae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:50.875324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66853637-7c84-4e12-a624-3313fe7acae7 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:50.885018 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1319/2679] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:51.207224 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12693860-b5e8-42f2-80bc-2b2a26e50bb9 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:51.208652 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-12693860-b5e8-42f2-80bc-2b2a26e50bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:51.223056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12693860-b5e8-42f2-80bc-2b2a26e50bb9 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:51.237678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:51.238640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:51.238930 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:51.239262 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d admin admin] PATH_INFO: `/v3/projects/c45f32fba7a14bf8aa83d330622c50a9` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:51.240568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c45f32fba7a14bf8aa83d330622c50a9)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:51.242578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:51.301845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c45f32fba7a14bf8aa83d330622c50a9', 'request_id': 'req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:51.310293 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c45f32fba7a14bf8aa83d330622c50a9', 'request_id': 'req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:51.318457 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c45f32fba7a14bf8aa83d330622c50a9', 'request_id': 'req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:51.333914 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d admin admin] The token cache is being invalidate because project c45f32fba7a14bf8aa83d330622c50a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:51.334218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:51.334850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:51.336360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1361/2680] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:51 2022] DELETE /identity/v3/projects/c45f32fba7a14bf8aa83d330622c50a9 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:51.340966 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66853637-7c84-4e12-a624-3313fe7acae7 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:51.351456 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66853637-7c84-4e12-a624-3313fe7acae7 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:51.436300 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdefb78-b497-4f5b-a581-4cb44e5de700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:51.437557 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdefb78-b497-4f5b-a581-4cb44e5de700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:51.437844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdefb78-b497-4f5b-a581-4cb44e5de700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:51.438175 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdefb78-b497-4f5b-a581-4cb44e5de700 admin admin] PATH_INFO: `/v3/users/591d4cfdf6c54654ac9130594c67cb98` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:51.439427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdefb78-b497-4f5b-a581-4cb44e5de700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=591d4cfdf6c54654ac9130594c67cb98)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:51.441442 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdefb78-b497-4f5b-a581-4cb44e5de700 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:51.443383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:51.516865 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ffdefb78-b497-4f5b-a581-4cb44e5de700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '591d4cfdf6c54654ac9130594c67cb98'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:51.534305 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ffdefb78-b497-4f5b-a581-4cb44e5de700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '591d4cfdf6c54654ac9130594c67cb98'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:51.535026 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ffdefb78-b497-4f5b-a581-4cb44e5de700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '591d4cfdf6c54654ac9130594c67cb98'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:51.535642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ffdefb78-b497-4f5b-a581-4cb44e5de700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '591d4cfdf6c54654ac9130594c67cb98'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:51.542845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ffdefb78-b497-4f5b-a581-4cb44e5de700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '591d4cfdf6c54654ac9130594c67cb98'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:51.557895 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1320/2681] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:51 2022] DELETE /identity/v3/users/591d4cfdf6c54654ac9130594c67cb98 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:51.852766 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:51.858864 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:51.878909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1208b08-9d6f-4411-92f0-7b9dcbe5ca4d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:51.947320 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:51.948383 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:51.948845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:51.949126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95 admin admin] PATH_INFO: `/v3/projects/0812cb6fe1e04c57a1e9341a484a0846` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:51.951619 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0812cb6fe1e04c57a1e9341a484a0846)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:51.954043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:51.971040 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdefb78-b497-4f5b-a581-4cb44e5de700 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:51.972527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ffdefb78-b497-4f5b-a581-4cb44e5de700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:51.986810 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdefb78-b497-4f5b-a581-4cb44e5de700 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:51.992607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536011ab-2ce5-4fc1-8555-7dca88f12c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:51.993670 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536011ab-2ce5-4fc1-8555-7dca88f12c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:51.993957 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536011ab-2ce5-4fc1-8555-7dca88f12c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:51.994242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536011ab-2ce5-4fc1-8555-7dca88f12c78 admin admin] PATH_INFO: `/v3/users/ddb5e1dec1904ae183e131ba95ed1ef2` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:51.996078 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536011ab-2ce5-4fc1-8555-7dca88f12c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddb5e1dec1904ae183e131ba95ed1ef2)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:51.998798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536011ab-2ce5-4fc1-8555-7dca88f12c78 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:51.998959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:52.020210 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0812cb6fe1e04c57a1e9341a484a0846', 'request_id': 'req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.024103 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0812cb6fe1e04c57a1e9341a484a0846', 'request_id': 'req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.031705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0812cb6fe1e04c57a1e9341a484a0846', 'request_id': 'req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.044144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95 admin admin] The token cache is being invalidate because project 0812cb6fe1e04c57a1e9341a484a0846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:52.044459 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.045085 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.046912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1362/2682] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:51 2022] DELETE /identity/v3/projects/0812cb6fe1e04c57a1e9341a484a0846 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:52.050168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-536011ab-2ce5-4fc1-8555-7dca88f12c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddb5e1dec1904ae183e131ba95ed1ef2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.050705 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:52.052083 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:52.067384 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-536011ab-2ce5-4fc1-8555-7dca88f12c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddb5e1dec1904ae183e131ba95ed1ef2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.068455 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-536011ab-2ce5-4fc1-8555-7dca88f12c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddb5e1dec1904ae183e131ba95ed1ef2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.069067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-536011ab-2ce5-4fc1-8555-7dca88f12c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddb5e1dec1904ae183e131ba95ed1ef2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.079038 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-536011ab-2ce5-4fc1-8555-7dca88f12c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddb5e1dec1904ae183e131ba95ed1ef2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.094392 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1321/2683] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:51 2022] DELETE /identity/v3/users/ddb5e1dec1904ae183e131ba95ed1ef2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:52.140339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4181c-6651-4ef2-87d5-d1af8f93cb95 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:52.205341 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c268a1-72ff-446c-89be-64c2f76e853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:52.206553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c268a1-72ff-446c-89be-64c2f76e853a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:52.207014 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c268a1-72ff-446c-89be-64c2f76e853a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:52.207574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c268a1-72ff-446c-89be-64c2f76e853a admin admin] PATH_INFO: `/v3/users/a73d43f0815b480799e6d55d436cecc6` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:52.210115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c268a1-72ff-446c-89be-64c2f76e853a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a73d43f0815b480799e6d55d436cecc6)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:52.212265 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c268a1-72ff-446c-89be-64c2f76e853a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:52.277360 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-72c268a1-72ff-446c-89be-64c2f76e853a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a73d43f0815b480799e6d55d436cecc6'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.287986 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536011ab-2ce5-4fc1-8555-7dca88f12c78 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:52.289948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-536011ab-2ce5-4fc1-8555-7dca88f12c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:52.301842 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-72c268a1-72ff-446c-89be-64c2f76e853a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a73d43f0815b480799e6d55d436cecc6'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.303428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-72c268a1-72ff-446c-89be-64c2f76e853a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a73d43f0815b480799e6d55d436cecc6'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.305553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:52.315451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-72c268a1-72ff-446c-89be-64c2f76e853a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a73d43f0815b480799e6d55d436cecc6'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.316702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-72c268a1-72ff-446c-89be-64c2f76e853a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a73d43f0815b480799e6d55d436cecc6'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.317029 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536011ab-2ce5-4fc1-8555-7dca88f12c78 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:52.325986 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac825943-9864-42da-add6-53a33c169ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:52.327426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac825943-9864-42da-add6-53a33c169ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:52.327426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac825943-9864-42da-add6-53a33c169ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:52.328229 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac825943-9864-42da-add6-53a33c169ccb admin admin] PATH_INFO: `/v3/projects/9bf94c44535545689ddc74bc82888a46` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:52.329762 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac825943-9864-42da-add6-53a33c169ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf94c44535545689ddc74bc82888a46)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:52.332099 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac825943-9864-42da-add6-53a33c169ccb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:52.338274 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1363/2684] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:52 2022] DELETE /identity/v3/users/a73d43f0815b480799e6d55d436cecc6 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:52.381980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=75,keystone:DELETE=23,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:52.385422 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ac825943-9864-42da-add6-53a33c169ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf94c44535545689ddc74bc82888a46', 'request_id': 'req-ac825943-9864-42da-add6-53a33c169ccb'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.395174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ac825943-9864-42da-add6-53a33c169ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf94c44535545689ddc74bc82888a46', 'request_id': 'req-ac825943-9864-42da-add6-53a33c169ccb'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.401323 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ac825943-9864-42da-add6-53a33c169ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf94c44535545689ddc74bc82888a46', 'request_id': 'req-ac825943-9864-42da-add6-53a33c169ccb'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.419279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ac825943-9864-42da-add6-53a33c169ccb admin admin] The token cache is being invalidate because project 9bf94c44535545689ddc74bc82888a46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:52.419627 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ac825943-9864-42da-add6-53a33c169ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.420299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ac825943-9864-42da-add6-53a33c169ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.425435 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1322/2685] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:52 2022] DELETE /identity/v3/projects/9bf94c44535545689ddc74bc82888a46 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:52.427334 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7c395-b6b7-4f70-b0bc-3cdde07f11cb None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:52.427622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7c395-b6b7-4f70-b0bc-3cdde07f11cb None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:52.427893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7c395-b6b7-4f70-b0bc-3cdde07f11cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:52.449184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-d7b7c395-b6b7-4f70-b0bc-3cdde07f11cb None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:52.451191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d7b7c395-b6b7-4f70-b0bc-3cdde07f11cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:52.523588 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1364/2686] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:52.533351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b532a-0617-40f2-98c4-7a9d9f74e80e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:52.533351 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b532a-0617-40f2-98c4-7a9d9f74e80e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:52.533814 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b532a-0617-40f2-98c4-7a9d9f74e80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:52.556055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-011b532a-0617-40f2-98c4-7a9d9f74e80e None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:52.557406 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-011b532a-0617-40f2-98c4-7a9d9f74e80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:52.572265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1323/2687] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:52.578201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7c395-b6b7-4f70-b0bc-3cdde07f11cb None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:52.589166 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7c395-b6b7-4f70-b0bc-3cdde07f11cb None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:52.595476 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015dc89-baab-4ce5-bfe0-570ed01ed29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:52.596144 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015dc89-baab-4ce5-bfe0-570ed01ed29e admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:52.596425 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015dc89-baab-4ce5-bfe0-570ed01ed29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:52.596698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015dc89-baab-4ce5-bfe0-570ed01ed29e admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:52.597479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015dc89-baab-4ce5-bfe0-570ed01ed29e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:37:52.597805 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015dc89-baab-4ce5-bfe0-570ed01ed29e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:52.600776 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015dc89-baab-4ce5-bfe0-570ed01ed29e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:52.615865 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1365/2688] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:37:52.620301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b532a-0617-40f2-98c4-7a9d9f74e80e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:52.633064 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b532a-0617-40f2-98c4-7a9d9f74e80e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:52.640034 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b32de6-183e-45a7-8685-16c3dfc94c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:52.640833 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b32de6-183e-45a7-8685-16c3dfc94c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:52.641119 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b32de6-183e-45a7-8685-16c3dfc94c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:52.641442 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b32de6-183e-45a7-8685-16c3dfc94c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:52.642463 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b32de6-183e-45a7-8685-16c3dfc94c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:52.644576 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b32de6-183e-45a7-8685-16c3dfc94c8d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:52.672535 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015dc89-baab-4ce5-bfe0-570ed01ed29e admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:52.674969 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-8015dc89-baab-4ce5-bfe0-570ed01ed29e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:52.683496 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1324/2689] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:52 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:52.688338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b32de6-183e-45a7-8685-16c3dfc94c8d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:52.695170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015dc89-baab-4ce5-bfe0-570ed01ed29e admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:52.760212 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599a0c5-7519-4e5f-b316-f691f67fa30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:52.761087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599a0c5-7519-4e5f-b316-f691f67fa30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:52.761401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599a0c5-7519-4e5f-b316-f691f67fa30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:52.761680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599a0c5-7519-4e5f-b316-f691f67fa30e admin admin] PATH_INFO: `/v3/projects/daf9d9082de9478a9a49676371a5a67f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:52.764427 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599a0c5-7519-4e5f-b316-f691f67fa30e admin admin] RBAC: Authorizing `identity:delete_project(project_id=daf9d9082de9478a9a49676371a5a67f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:52.768763 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599a0c5-7519-4e5f-b316-f691f67fa30e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:52.771679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b32de6-183e-45a7-8685-16c3dfc94c8d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:52.776750 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa847653-2d14-4039-8436-e170d97a6011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:52.777487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa847653-2d14-4039-8436-e170d97a6011 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:52.778320 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa847653-2d14-4039-8436-e170d97a6011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:52.778757 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa847653-2d14-4039-8436-e170d97a6011 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:52.779495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa847653-2d14-4039-8436-e170d97a6011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:52.781547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa847653-2d14-4039-8436-e170d97a6011 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:52.806201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1325/2690] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:52 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:37:52.811007 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa847653-2d14-4039-8436-e170d97a6011 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:52.825774 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa847653-2d14-4039-8436-e170d97a6011 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:52.832259 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-2599a0c5-7519-4e5f-b316-f691f67fa30e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daf9d9082de9478a9a49676371a5a67f', 'request_id': 'req-2599a0c5-7519-4e5f-b316-f691f67fa30e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.839009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-2599a0c5-7519-4e5f-b316-f691f67fa30e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daf9d9082de9478a9a49676371a5a67f', 'request_id': 'req-2599a0c5-7519-4e5f-b316-f691f67fa30e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.849022 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-2599a0c5-7519-4e5f-b316-f691f67fa30e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daf9d9082de9478a9a49676371a5a67f', 'request_id': 'req-2599a0c5-7519-4e5f-b316-f691f67fa30e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.859651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:52.863484 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-2599a0c5-7519-4e5f-b316-f691f67fa30e admin admin] The token cache is being invalidate because project daf9d9082de9478a9a49676371a5a67f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:52.864366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-2599a0c5-7519-4e5f-b316-f691f67fa30e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.865685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-2599a0c5-7519-4e5f-b316-f691f67fa30e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:52.867651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1366/2691] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:52 2022] DELETE /identity/v3/projects/daf9d9082de9478a9a49676371a5a67f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:52.873734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef726e3-e9f1-4911-b3e8-27ac0cafb45a None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:52.874242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef726e3-e9f1-4911-b3e8-27ac0cafb45a None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:52.874515 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef726e3-e9f1-4911-b3e8-27ac0cafb45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:52.893519 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-eef726e3-e9f1-4911-b3e8-27ac0cafb45a None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:52.894754 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-eef726e3-e9f1-4911-b3e8-27ac0cafb45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:52.895796 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27860d8c-acf5-41ef-8110-65d7ed577a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:52.896477 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27860d8c-acf5-41ef-8110-65d7ed577a5c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:52.896760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27860d8c-acf5-41ef-8110-65d7ed577a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:52.897115 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27860d8c-acf5-41ef-8110-65d7ed577a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:52.897804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27860d8c-acf5-41ef-8110-65d7ed577a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:52.902331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27860d8c-acf5-41ef-8110-65d7ed577a5c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:52.904839 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1367/2692] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:52.914816 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3009d-6a58-4068-bd8a-9314e50d2831 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:52.916374 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3009d-6a58-4068-bd8a-9314e50d2831 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:52.916374 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3009d-6a58-4068-bd8a-9314e50d2831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:52.918065 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1326/2693] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:37:52.923612 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27860d8c-acf5-41ef-8110-65d7ed577a5c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:52.925329 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-27860d8c-acf5-41ef-8110-65d7ed577a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:52.936109 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-e0d3009d-6a58-4068-bd8a-9314e50d2831 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:52.938146 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e0d3009d-6a58-4068-bd8a-9314e50d2831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:52.939821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27860d8c-acf5-41ef-8110-65d7ed577a5c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:52.944515 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df1f35-7e81-41ec-be9d-8c8e4e3a3f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:52.945227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df1f35-7e81-41ec-be9d-8c8e4e3a3f68 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:52.945517 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df1f35-7e81-41ec-be9d-8c8e4e3a3f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:52.945796 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df1f35-7e81-41ec-be9d-8c8e4e3a3f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:52.946528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df1f35-7e81-41ec-be9d-8c8e4e3a3f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:52.947605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1368/2694] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:52.948155 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df1f35-7e81-41ec-be9d-8c8e4e3a3f68 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:52.952924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3009d-6a58-4068-bd8a-9314e50d2831 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:52.963474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1327/2695] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:37:52.963958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3009d-6a58-4068-bd8a-9314e50d2831 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:52.964474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df1f35-7e81-41ec-be9d-8c8e4e3a3f68 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:52.970244 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c165826-23cf-4907-8349-fcb28aac0e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:52.970911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c165826-23cf-4907-8349-fcb28aac0e2e admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:52.971190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c165826-23cf-4907-8349-fcb28aac0e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:52.971473 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c165826-23cf-4907-8349-fcb28aac0e2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:52.972455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c165826-23cf-4907-8349-fcb28aac0e2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:37:52.972696 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c165826-23cf-4907-8349-fcb28aac0e2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:52.974486 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c165826-23cf-4907-8349-fcb28aac0e2e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:52.976041 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df1f35-7e81-41ec-be9d-8c8e4e3a3f68 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:52.983748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16362bd2-8ef8-4ad3-8cbd-5f16b47af095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:52.984663 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16362bd2-8ef8-4ad3-8cbd-5f16b47af095 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:52.984947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16362bd2-8ef8-4ad3-8cbd-5f16b47af095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:52.985248 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16362bd2-8ef8-4ad3-8cbd-5f16b47af095 admin admin] PATH_INFO: `/v3/projects/a767c10ce33549568e400d584e1c08f8/users/15ed522956fa457eabbec8d5f48924f0/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:52.987301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1369/2696] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:37:52.991807 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c165826-23cf-4907-8349-fcb28aac0e2e admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:53.002157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c165826-23cf-4907-8349-fcb28aac0e2e admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:53.002511 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16362bd2-8ef8-4ad3-8cbd-5f16b47af095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a767c10ce33549568e400d584e1c08f8, user_id=15ed522956fa457eabbec8d5f48924f0, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:53.005511 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16362bd2-8ef8-4ad3-8cbd-5f16b47af095 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:53.007424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf5393-1472-4090-9555-480cafc7e797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:53.008093 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf5393-1472-4090-9555-480cafc7e797 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.008374 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf5393-1472-4090-9555-480cafc7e797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.008653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf5393-1472-4090-9555-480cafc7e797 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.009172 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=4,keystone:SELECT=90,keystone:INSERT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:53.009518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf5393-1472-4090-9555-480cafc7e797 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:53.012634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf5393-1472-4090-9555-480cafc7e797 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:53.024788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1328/2697] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:52 2022] PUT /identity/v3/projects/a767c10ce33549568e400d584e1c08f8/users/15ed522956fa457eabbec8d5f48924f0/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:53.036475 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b65518-1432-469b-a040-ebc6b0ebf9c0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.036855 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b65518-1432-469b-a040-ebc6b0ebf9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.037286 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b65518-1432-469b-a040-ebc6b0ebf9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.050673 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1370/2698] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:52 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:37:53.051581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf5393-1472-4090-9555-480cafc7e797 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:53.068444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b6b65518-1432-469b-a040-ebc6b0ebf9c0 None None] MFA Rules not processed for user `15ed522956fa457eabbec8d5f48924f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:53.069720 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b6b65518-1432-469b-a040-ebc6b0ebf9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:53.126526 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf5393-1472-4090-9555-480cafc7e797 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:53.131583 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b5bf0-4023-4024-89b3-389690859bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:53.132337 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b5bf0-4023-4024-89b3-389690859bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.132633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b5bf0-4023-4024-89b3-389690859bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.132909 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b5bf0-4023-4024-89b3-389690859bc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.133657 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b5bf0-4023-4024-89b3-389690859bc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:53.135664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b5bf0-4023-4024-89b3-389690859bc6 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:53.146217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1329/2699] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:53.158594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b65518-1432-469b-a040-ebc6b0ebf9c0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:53.160280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1371/2700] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:53 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:37:53.160437 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b6b65518-1432-469b-a040-ebc6b0ebf9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:53.164401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b5bf0-4023-4024-89b3-389690859bc6 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:53.174718 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b5bf0-4023-4024-89b3-389690859bc6 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:53.180001 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2b7db-37be-4ead-aa40-60261daab3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:53.180664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2b7db-37be-4ead-aa40-60261daab3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.182393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2b7db-37be-4ead-aa40-60261daab3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.182686 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2b7db-37be-4ead-aa40-60261daab3e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.183428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2b7db-37be-4ead-aa40-60261daab3e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:53.185055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2b7db-37be-4ead-aa40-60261daab3e5 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:53.197728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1372/2701] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:37:53.202286 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2b7db-37be-4ead-aa40-60261daab3e5 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:53.213790 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2b7db-37be-4ead-aa40-60261daab3e5 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:53.219774 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b97134-ece3-4481-b99d-959a75bd6b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:53.220845 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b97134-ece3-4481-b99d-959a75bd6b09 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.221317 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b97134-ece3-4481-b99d-959a75bd6b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.221600 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b97134-ece3-4481-b99d-959a75bd6b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.222415 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b97134-ece3-4481-b99d-959a75bd6b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:53.225414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b97134-ece3-4481-b99d-959a75bd6b09 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:53.240190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1373/2702] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:37:53.245016 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b97134-ece3-4481-b99d-959a75bd6b09 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:53.253411 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b65518-1432-469b-a040-ebc6b0ebf9c0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:53.260462 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b97134-ece3-4481-b99d-959a75bd6b09 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:53.261041 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9958ca-d735-435c-bbf6-17cadbd42310 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:53.261751 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9958ca-d735-435c-bbf6-17cadbd42310 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.262617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9958ca-d735-435c-bbf6-17cadbd42310 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.262947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9958ca-d735-435c-bbf6-17cadbd42310 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.266071 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20ff76-4277-46f2-a376-dedc9bbea24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:53.267006 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20ff76-4277-46f2-a376-dedc9bbea24b admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.267556 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20ff76-4277-46f2-a376-dedc9bbea24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.267924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20ff76-4277-46f2-a376-dedc9bbea24b admin admin] PATH_INFO: `/v3/projects/6f64c59d39b9480bab8f007fbb1dac5f/users/1cadf8f5d109489189c0d4e299f57d29/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.271852 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9958ca-d735-435c-bbf6-17cadbd42310 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:53.273482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9958ca-d735-435c-bbf6-17cadbd42310 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:53.283283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1330/2703] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:37:53.283557 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20ff76-4277-46f2-a376-dedc9bbea24b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f64c59d39b9480bab8f007fbb1dac5f, user_id=1cadf8f5d109489189c0d4e299f57d29, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:53.286376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20ff76-4277-46f2-a376-dedc9bbea24b admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:53.300397 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1374/2704] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:53 2022] PUT /identity/v3/projects/6f64c59d39b9480bab8f007fbb1dac5f/users/1cadf8f5d109489189c0d4e299f57d29/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:53.370047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8808fc-e6d2-4d6c-b7cd-d2157c44752b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.370420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8808fc-e6d2-4d6c-b7cd-d2157c44752b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.370767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8808fc-e6d2-4d6c-b7cd-d2157c44752b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.406535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9c8808fc-e6d2-4d6c-b7cd-d2157c44752b None None] MFA Rules not processed for user `1cadf8f5d109489189c0d4e299f57d29`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:53.407763 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9c8808fc-e6d2-4d6c-b7cd-d2157c44752b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:53.485369 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:53.493124 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1331/2705] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:53.494380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20ff76-4277-46f2-a376-dedc9bbea24b admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:53.518432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8808fc-e6d2-4d6c-b7cd-d2157c44752b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:53.561761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:53.568061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20ff76-4277-46f2-a376-dedc9bbea24b admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:53.573391 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2b1ab-7316-4e4a-8d04-d600296f296d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:53.574063 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2b1ab-7316-4e4a-8d04-d600296f296d admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.574349 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2b1ab-7316-4e4a-8d04-d600296f296d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.574800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2b1ab-7316-4e4a-8d04-d600296f296d admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.575498 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2b1ab-7316-4e4a-8d04-d600296f296d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:53.577419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2b1ab-7316-4e4a-8d04-d600296f296d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:53.585427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8808fc-e6d2-4d6c-b7cd-d2157c44752b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:53.590785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69ec3a-fa16-4238-a2ce-0072be98d950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:53.591655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69ec3a-fa16-4238-a2ce-0072be98d950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.591934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69ec3a-fa16-4238-a2ce-0072be98d950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.592586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69ec3a-fa16-4238-a2ce-0072be98d950 admin admin] PATH_INFO: `/v3/users/15ed522956fa457eabbec8d5f48924f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.593483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69ec3a-fa16-4238-a2ce-0072be98d950 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15ed522956fa457eabbec8d5f48924f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:53.596189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69ec3a-fa16-4238-a2ce-0072be98d950 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:53.604707 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1375/2706] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:53 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:53.609415 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2b1ab-7316-4e4a-8d04-d600296f296d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:53.645331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b69ec3a-fa16-4238-a2ce-0072be98d950 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15ed522956fa457eabbec8d5f48924f0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:53.658583 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b69ec3a-fa16-4238-a2ce-0072be98d950 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15ed522956fa457eabbec8d5f48924f0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:53.659299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b69ec3a-fa16-4238-a2ce-0072be98d950 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15ed522956fa457eabbec8d5f48924f0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:53.659921 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b69ec3a-fa16-4238-a2ce-0072be98d950 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15ed522956fa457eabbec8d5f48924f0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:53.667109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b69ec3a-fa16-4238-a2ce-0072be98d950 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15ed522956fa457eabbec8d5f48924f0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:53.674391 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2b1ab-7316-4e4a-8d04-d600296f296d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:53.679569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3d3e8-6a50-41c5-b333-82ec79d9d4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:53.680235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3d3e8-6a50-41c5-b333-82ec79d9d4ce admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.680520 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3d3e8-6a50-41c5-b333-82ec79d9d4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.680812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3d3e8-6a50-41c5-b333-82ec79d9d4ce admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.681399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1332/2707] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:53 2022] DELETE /identity/v3/users/15ed522956fa457eabbec8d5f48924f0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:53.681537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3d3e8-6a50-41c5-b333-82ec79d9d4ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:53.683426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3d3e8-6a50-41c5-b333-82ec79d9d4ce admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:53.685944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498e3e5-14d5-43c8-b60e-56a0e93b00c4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.686234 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498e3e5-14d5-43c8-b60e-56a0e93b00c4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.686528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498e3e5-14d5-43c8-b60e-56a0e93b00c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.704511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1376/2708] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:53 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:37:53.706201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c498e3e5-14d5-43c8-b60e-56a0e93b00c4 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:53.707508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c498e3e5-14d5-43c8-b60e-56a0e93b00c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:53.708574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3d3e8-6a50-41c5-b333-82ec79d9d4ce admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:53.710140 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c9d3d3e8-6a50-41c5-b333-82ec79d9d4ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:53.722389 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3d3e8-6a50-41c5-b333-82ec79d9d4ce admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:53.770708 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1333/2709] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:53.779870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe386e-e725-499c-9da4-b87c2380a437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:53.780533 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe386e-e725-499c-9da4-b87c2380a437 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.780819 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe386e-e725-499c-9da4-b87c2380a437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.781091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe386e-e725-499c-9da4-b87c2380a437 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.781210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498e3e5-14d5-43c8-b60e-56a0e93b00c4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:53.781856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe386e-e725-499c-9da4-b87c2380a437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:53.782524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c498e3e5-14d5-43c8-b60e-56a0e93b00c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:53.783452 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe386e-e725-499c-9da4-b87c2380a437 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:53.794242 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1377/2710] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:53.798570 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe386e-e725-499c-9da4-b87c2380a437 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:53.806472 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:53.808606 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe386e-e725-499c-9da4-b87c2380a437 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:53.813670 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a554255-0a59-4b2e-b917-133968949b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:53.814326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a554255-0a59-4b2e-b917-133968949b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.814608 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a554255-0a59-4b2e-b917-133968949b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.814888 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a554255-0a59-4b2e-b917-133968949b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.815732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a554255-0a59-4b2e-b917-133968949b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:53.817316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a554255-0a59-4b2e-b917-133968949b36 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:53.828205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1378/2711] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:37:53.832191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a554255-0a59-4b2e-b917-133968949b36 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:53.843102 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a554255-0a59-4b2e-b917-133968949b36 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:53.848002 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8bbf8-141b-434f-b5d0-71c75e9c5d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:53.848908 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8bbf8-141b-434f-b5d0-71c75e9c5d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.849238 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8bbf8-141b-434f-b5d0-71c75e9c5d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.849511 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8bbf8-141b-434f-b5d0-71c75e9c5d7a admin admin] PATH_INFO: `/v3/projects/e168c6c648b746c3a9bef7219a6ddd2d/users/2ccc27d2267e437d8f1c15a59a6ba65b/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.862281 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498e3e5-14d5-43c8-b60e-56a0e93b00c4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:53.864308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8bbf8-141b-434f-b5d0-71c75e9c5d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e168c6c648b746c3a9bef7219a6ddd2d, user_id=2ccc27d2267e437d8f1c15a59a6ba65b, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:53.866129 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf6b26-9202-4720-b37d-0a2c2a0f95fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:53.866785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf6b26-9202-4720-b37d-0a2c2a0f95fc service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.866993 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8bbf8-141b-434f-b5d0-71c75e9c5d7a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:53.867127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf6b26-9202-4720-b37d-0a2c2a0f95fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.867330 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf6b26-9202-4720-b37d-0a2c2a0f95fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.877806 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf6b26-9202-4720-b37d-0a2c2a0f95fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:53.879320 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf6b26-9202-4720-b37d-0a2c2a0f95fc service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:53.880077 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1379/2712] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:53 2022] PUT /identity/v3/projects/e168c6c648b746c3a9bef7219a6ddd2d/users/2ccc27d2267e437d8f1c15a59a6ba65b/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:53.890161 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1334/2713] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:37:53.890829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36400208-f5fb-4223-b36e-a317defdfa97 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:53.891115 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36400208-f5fb-4223-b36e-a317defdfa97 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:53.891394 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36400208-f5fb-4223-b36e-a317defdfa97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:53.918413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-36400208-f5fb-4223-b36e-a317defdfa97 None None] MFA Rules not processed for user `2ccc27d2267e437d8f1c15a59a6ba65b`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:53.919702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-36400208-f5fb-4223-b36e-a317defdfa97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:53.940098 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:53.992153 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1380/2714] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:53.996776 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf6b26-9202-4720-b37d-0a2c2a0f95fc service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:54.006535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf6b26-9202-4720-b37d-0a2c2a0f95fc service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:54.067650 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1dad45-5fcc-4ff7-8948-8a8f15875725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:54.068318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1dad45-5fcc-4ff7-8948-8a8f15875725 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.068608 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1dad45-5fcc-4ff7-8948-8a8f15875725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.068883 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1dad45-5fcc-4ff7-8948-8a8f15875725 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.069625 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1dad45-5fcc-4ff7-8948-8a8f15875725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:54.071527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1dad45-5fcc-4ff7-8948-8a8f15875725 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:54.103929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1335/2715] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:53 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:54.105188 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36400208-f5fb-4223-b36e-a317defdfa97 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:54.116674 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36400208-f5fb-4223-b36e-a317defdfa97 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:54.178168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1dad45-5fcc-4ff7-8948-8a8f15875725 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:54.180239 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4b1dad45-5fcc-4ff7-8948-8a8f15875725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:54.187366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8562b-294b-45a5-b27b-8b6bb006de30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:54.188055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8562b-294b-45a5-b27b-8b6bb006de30 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.188339 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8562b-294b-45a5-b27b-8b6bb006de30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.188617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8562b-294b-45a5-b27b-8b6bb006de30 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.189350 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8562b-294b-45a5-b27b-8b6bb006de30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:54.191350 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8562b-294b-45a5-b27b-8b6bb006de30 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:54.214490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1381/2716] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:54 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:54.218964 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8562b-294b-45a5-b27b-8b6bb006de30 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:54.229350 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8562b-294b-45a5-b27b-8b6bb006de30 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:54.234716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dff30-7a0e-414a-b8f8-7e09f0a7be3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:54.235406 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dff30-7a0e-414a-b8f8-7e09f0a7be3b admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.235689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dff30-7a0e-414a-b8f8-7e09f0a7be3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.235967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dff30-7a0e-414a-b8f8-7e09f0a7be3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.237207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dff30-7a0e-414a-b8f8-7e09f0a7be3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:54.238837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dff30-7a0e-414a-b8f8-7e09f0a7be3b admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:54.255207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1382/2717] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:37:54.259372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dff30-7a0e-414a-b8f8-7e09f0a7be3b admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:54.260871 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1dad45-5fcc-4ff7-8948-8a8f15875725 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:54.266752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1829d3-99ed-47d3-b31c-aeeadc02f560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:54.268043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1829d3-99ed-47d3-b31c-aeeadc02f560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.268327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1829d3-99ed-47d3-b31c-aeeadc02f560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.268706 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1829d3-99ed-47d3-b31c-aeeadc02f560 admin admin] PATH_INFO: `/v3/projects/a767c10ce33549568e400d584e1c08f8` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.269655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dff30-7a0e-414a-b8f8-7e09f0a7be3b admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:54.269930 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1829d3-99ed-47d3-b31c-aeeadc02f560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a767c10ce33549568e400d584e1c08f8)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:54.271829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1829d3-99ed-47d3-b31c-aeeadc02f560 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:54.274194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4fc27-fae3-45a8-b655-4995abb7ebfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:54.275059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4fc27-fae3-45a8-b655-4995abb7ebfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.275342 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4fc27-fae3-45a8-b655-4995abb7ebfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.275619 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4fc27-fae3-45a8-b655-4995abb7ebfb admin admin] PATH_INFO: `/v3/projects/9e986ceffc2a47e28052eba6a20af6f4/users/fec132f9fbe64cdf804061a9f9e39a8e/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.295276 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4fc27-fae3-45a8-b655-4995abb7ebfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e986ceffc2a47e28052eba6a20af6f4, user_id=fec132f9fbe64cdf804061a9f9e39a8e, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:54.298930 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4fc27-fae3-45a8-b655-4995abb7ebfb admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:54.312518 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1383/2718] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:54 2022] PUT /identity/v3/projects/9e986ceffc2a47e28052eba6a20af6f4/users/fec132f9fbe64cdf804061a9f9e39a8e/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:54.316317 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4fc27-fae3-45a8-b655-4995abb7ebfb admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:54.322368 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8f1829d3-99ed-47d3-b31c-aeeadc02f560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a767c10ce33549568e400d584e1c08f8', 'request_id': 'req-8f1829d3-99ed-47d3-b31c-aeeadc02f560'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:54.324054 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:54.331390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8f1829d3-99ed-47d3-b31c-aeeadc02f560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a767c10ce33549568e400d584e1c08f8', 'request_id': 'req-8f1829d3-99ed-47d3-b31c-aeeadc02f560'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:54.338648 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8f1829d3-99ed-47d3-b31c-aeeadc02f560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a767c10ce33549568e400d584e1c08f8', 'request_id': 'req-8f1829d3-99ed-47d3-b31c-aeeadc02f560'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:54.351481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8f1829d3-99ed-47d3-b31c-aeeadc02f560 admin admin] The token cache is being invalidate because project a767c10ce33549568e400d584e1c08f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:37:54.351820 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8f1829d3-99ed-47d3-b31c-aeeadc02f560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:54.352482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8f1829d3-99ed-47d3-b31c-aeeadc02f560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:54.354059 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1336/2719] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:37:54 2022] DELETE /identity/v3/projects/a767c10ce33549568e400d584e1c08f8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:54.359246 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ca63e-02c4-40a1-95a7-74759a60e060 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.359531 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ca63e-02c4-40a1-95a7-74759a60e060 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.359809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ca63e-02c4-40a1-95a7-74759a60e060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.378753 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-692ca63e-02c4-40a1-95a7-74759a60e060 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:54.379981 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-692ca63e-02c4-40a1-95a7-74759a60e060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:54.381385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4fc27-fae3-45a8-b655-4995abb7ebfb admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:54.399677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:54.442416 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1337/2720] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:54.442898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e8525-ac41-4932-8fbc-ba53092669bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:54.443570 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e8525-ac41-4932-8fbc-ba53092669bc admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.443859 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e8525-ac41-4932-8fbc-ba53092669bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.444138 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e8525-ac41-4932-8fbc-ba53092669bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.444876 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e8525-ac41-4932-8fbc-ba53092669bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:54.446548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e8525-ac41-4932-8fbc-ba53092669bc admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:54.451818 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb5156-5290-466e-ab5a-8fbd32829462 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.452113 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb5156-5290-466e-ab5a-8fbd32829462 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.452394 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb5156-5290-466e-ab5a-8fbd32829462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.460759 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1384/2721] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:37:54.461762 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e8525-ac41-4932-8fbc-ba53092669bc admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:54.463470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5e9e8525-ac41-4932-8fbc-ba53092669bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:54.474010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bdcb5156-5290-466e-ab5a-8fbd32829462 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:54.475205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bdcb5156-5290-466e-ab5a-8fbd32829462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:54.477736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e8525-ac41-4932-8fbc-ba53092669bc admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:54.482879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7553eee-e5b2-4161-a194-91eaac6449cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:54.483813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7553eee-e5b2-4161-a194-91eaac6449cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.484098 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7553eee-e5b2-4161-a194-91eaac6449cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.484376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7553eee-e5b2-4161-a194-91eaac6449cc admin admin] PATH_INFO: `/v3/projects/9e986ceffc2a47e28052eba6a20af6f4/users/fec132f9fbe64cdf804061a9f9e39a8e/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.484956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1338/2722] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:54.486482 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7553eee-e5b2-4161-a194-91eaac6449cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e986ceffc2a47e28052eba6a20af6f4, user_id=fec132f9fbe64cdf804061a9f9e39a8e, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:54.489007 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb5156-5290-466e-ab5a-8fbd32829462 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:54.489160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7553eee-e5b2-4161-a194-91eaac6449cc admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:54.498145 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb5156-5290-466e-ab5a-8fbd32829462 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:54.500082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1385/2723] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:54 2022] PUT /identity/v3/projects/9e986ceffc2a47e28052eba6a20af6f4/users/fec132f9fbe64cdf804061a9f9e39a8e/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:54.510105 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a968da-789b-4bd1-b786-d8198412b52e None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.510397 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a968da-789b-4bd1-b786-d8198412b52e None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.510680 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a968da-789b-4bd1-b786-d8198412b52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.538495 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-43a968da-789b-4bd1-b786-d8198412b52e None None] MFA Rules not processed for user `fec132f9fbe64cdf804061a9f9e39a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:54.539709 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-43a968da-789b-4bd1-b786-d8198412b52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:54.555634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055c8c2-645d-4662-8d03-c3869ae91e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:54.556322 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055c8c2-645d-4662-8d03-c3869ae91e33 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.556604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055c8c2-645d-4662-8d03-c3869ae91e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.556882 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055c8c2-645d-4662-8d03-c3869ae91e33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.557641 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055c8c2-645d-4662-8d03-c3869ae91e33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:37:54.557954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055c8c2-645d-4662-8d03-c3869ae91e33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:54.559720 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055c8c2-645d-4662-8d03-c3869ae91e33 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:54.571030 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1339/2724] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:37:54.576548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055c8c2-645d-4662-8d03-c3869ae91e33 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:54.586721 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055c8c2-645d-4662-8d03-c3869ae91e33 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:54.592851 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb6936-8eea-4575-9b49-d7a6f5fb401b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:54.593589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb6936-8eea-4575-9b49-d7a6f5fb401b admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.593945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb6936-8eea-4575-9b49-d7a6f5fb401b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.594234 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb6936-8eea-4575-9b49-d7a6f5fb401b admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.594927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb6936-8eea-4575-9b49-d7a6f5fb401b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:54.596888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb6936-8eea-4575-9b49-d7a6f5fb401b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:54.610651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1386/2725] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:37:54.622850 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a968da-789b-4bd1-b786-d8198412b52e None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:54.626259 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1340/2726] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:54 2022] POST /identity/v3/projects => generated 401 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:37:54.626393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-43a968da-789b-4bd1-b786-d8198412b52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:54.627235 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb6936-8eea-4575-9b49-d7a6f5fb401b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:54.675221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:54.683983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a968da-789b-4bd1-b786-d8198412b52e None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:54.694807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb6936-8eea-4575-9b49-d7a6f5fb401b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:54.699971 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a64ea6-9cfc-4362-b826-fda4084dc308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:54.700680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a64ea6-9cfc-4362-b826-fda4084dc308 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.700968 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a64ea6-9cfc-4362-b826-fda4084dc308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.701285 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a64ea6-9cfc-4362-b826-fda4084dc308 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.701978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a64ea6-9cfc-4362-b826-fda4084dc308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:54.703871 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a64ea6-9cfc-4362-b826-fda4084dc308 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:54.728075 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1341/2727] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:54 2022] POST /identity/v3/users => generated 301 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:54.729501 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a64ea6-9cfc-4362-b826-fda4084dc308 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:54.730497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:54.740199 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a64ea6-9cfc-4362-b826-fda4084dc308 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:54.744913 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24dcc55-87c3-42c5-9d7d-95a40aa908ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:54.745615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24dcc55-87c3-42c5-9d7d-95a40aa908ff admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.745904 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24dcc55-87c3-42c5-9d7d-95a40aa908ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.746186 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24dcc55-87c3-42c5-9d7d-95a40aa908ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.747182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24dcc55-87c3-42c5-9d7d-95a40aa908ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:54.749497 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24dcc55-87c3-42c5-9d7d-95a40aa908ff admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:54.750442 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480fbf1c-472d-42d4-9671-9228cb6d8547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:54.751135 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480fbf1c-472d-42d4-9671-9228cb6d8547 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.751421 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480fbf1c-472d-42d4-9671-9228cb6d8547 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.751706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480fbf1c-472d-42d4-9671-9228cb6d8547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.754056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-480fbf1c-472d-42d4-9671-9228cb6d8547 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:54.766309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1342/2728] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:54.767712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24dcc55-87c3-42c5-9d7d-95a40aa908ff admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:54.778302 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24dcc55-87c3-42c5-9d7d-95a40aa908ff admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:54.783082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa78b5-a74d-445b-afc5-a603fcfba095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:54.783737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa78b5-a74d-445b-afc5-a603fcfba095 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.784018 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa78b5-a74d-445b-afc5-a603fcfba095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.784293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa78b5-a74d-445b-afc5-a603fcfba095 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.785012 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa78b5-a74d-445b-afc5-a603fcfba095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:54.786624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa78b5-a74d-445b-afc5-a603fcfba095 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:54.798764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1343/2729] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:54.802554 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa78b5-a74d-445b-afc5-a603fcfba095 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:54.812791 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa78b5-a74d-445b-afc5-a603fcfba095 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:54.817458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168b7da-ef35-4339-be50-139ef72891b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:54.818122 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480fbf1c-472d-42d4-9671-9228cb6d8547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:54.818425 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168b7da-ef35-4339-be50-139ef72891b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.818711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168b7da-ef35-4339-be50-139ef72891b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.818986 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168b7da-ef35-4339-be50-139ef72891b3 admin admin] PATH_INFO: `/v3/projects/e5341f7e995b4536abfc75d5e3295ffc/users/f5898343f15142ad96152fac7d86bbe2/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.819561 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480fbf1c-472d-42d4-9671-9228cb6d8547 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:54.829255 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1387/2730] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:37:54.836024 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168b7da-ef35-4339-be50-139ef72891b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5341f7e995b4536abfc75d5e3295ffc, user_id=f5898343f15142ad96152fac7d86bbe2, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:54.838714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168b7da-ef35-4339-be50-139ef72891b3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:54.853487 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1344/2731] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:54 2022] PUT /identity/v3/projects/e5341f7e995b4536abfc75d5e3295ffc/users/f5898343f15142ad96152fac7d86bbe2/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:54.864327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac3470-e212-46ba-b6b2-87e04adb11b7 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:54.864782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac3470-e212-46ba-b6b2-87e04adb11b7 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:54.864896 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac3470-e212-46ba-b6b2-87e04adb11b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:54.891262 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-7cac3470-e212-46ba-b6b2-87e04adb11b7 None None] MFA Rules not processed for user `f5898343f15142ad96152fac7d86bbe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:54.892753 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7cac3470-e212-46ba-b6b2-87e04adb11b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:54.962564 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1388/2732] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:54.967920 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168b7da-ef35-4339-be50-139ef72891b3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:55.039546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168b7da-ef35-4339-be50-139ef72891b3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:55.044494 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acac036-6374-4cc7-8703-3c789caeb9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:55.045171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acac036-6374-4cc7-8703-3c789caeb9b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:55.045495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acac036-6374-4cc7-8703-3c789caeb9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:55.045761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acac036-6374-4cc7-8703-3c789caeb9b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:55.046686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acac036-6374-4cc7-8703-3c789caeb9b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:55.048758 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acac036-6374-4cc7-8703-3c789caeb9b1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:55.090588 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1345/2733] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:37:54 2022] POST /identity/v3/projects => generated 401 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:55.094734 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac3470-e212-46ba-b6b2-87e04adb11b7 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:55.107759 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac3470-e212-46ba-b6b2-87e04adb11b7 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:55.171113 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ffad4-9ba8-49dc-a5d8-594747ecfc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:55.171846 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ffad4-9ba8-49dc-a5d8-594747ecfc75 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:55.172128 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ffad4-9ba8-49dc-a5d8-594747ecfc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:55.172410 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ffad4-9ba8-49dc-a5d8-594747ecfc75 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:55.173105 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ffad4-9ba8-49dc-a5d8-594747ecfc75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:55.175110 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ffad4-9ba8-49dc-a5d8-594747ecfc75 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:55.195832 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1389/2734] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:37:55 2022] POST /identity/v3/users => generated 307 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:37:55.199973 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acac036-6374-4cc7-8703-3c789caeb9b1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:55.210528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acac036-6374-4cc7-8703-3c789caeb9b1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:55.215941 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f5398-241a-4b42-9fa2-88ffec81188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:55.216549 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f5398-241a-4b42-9fa2-88ffec81188d admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:55.216833 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f5398-241a-4b42-9fa2-88ffec81188d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:55.217121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f5398-241a-4b42-9fa2-88ffec81188d admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:55.217880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f5398-241a-4b42-9fa2-88ffec81188d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:55.219498 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f5398-241a-4b42-9fa2-88ffec81188d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:55.230077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1346/2735] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:55.234973 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ffad4-9ba8-49dc-a5d8-594747ecfc75 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:55.245886 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ffad4-9ba8-49dc-a5d8-594747ecfc75 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:55.250875 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2215638-2721-4493-b414-3f5d739b8c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:55.251761 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215638-2721-4493-b414-3f5d739b8c0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:55.252045 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215638-2721-4493-b414-3f5d739b8c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:55.252317 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215638-2721-4493-b414-3f5d739b8c0c admin admin] PATH_INFO: `/v3/projects/04a66f7a7ec84974ae65e72e6f7ecb43/users/670cbd3e80a44d00b69504b98573fcb9/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:55.267817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2215638-2721-4493-b414-3f5d739b8c0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=04a66f7a7ec84974ae65e72e6f7ecb43, user_id=670cbd3e80a44d00b69504b98573fcb9, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:55.270489 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2215638-2721-4493-b414-3f5d739b8c0c admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:55.273505 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:55.285724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1390/2736] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:55 2022] PUT /identity/v3/projects/04a66f7a7ec84974ae65e72e6f7ecb43/users/670cbd3e80a44d00b69504b98573fcb9/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:55.289748 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f5398-241a-4b42-9fa2-88ffec81188d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:55.300559 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f5398-241a-4b42-9fa2-88ffec81188d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:55.356642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7b6d5-215a-48ec-acb5-c6b76adf5fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:55.357340 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7b6d5-215a-48ec-acb5-c6b76adf5fee admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:55.357624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7b6d5-215a-48ec-acb5-c6b76adf5fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:55.357900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7b6d5-215a-48ec-acb5-c6b76adf5fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:55.358620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7b6d5-215a-48ec-acb5-c6b76adf5fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:55.360268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7b6d5-215a-48ec-acb5-c6b76adf5fee admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:55.362629 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:55.372426 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1347/2737] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:37:55.376804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2215638-2721-4493-b414-3f5d739b8c0c admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:55.387540 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2215638-2721-4493-b414-3f5d739b8c0c admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:55.392470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab20fdf-a24c-4da7-90a8-22f57fee29b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:55.393377 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab20fdf-a24c-4da7-90a8-22f57fee29b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:55.393660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab20fdf-a24c-4da7-90a8-22f57fee29b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:55.393933 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab20fdf-a24c-4da7-90a8-22f57fee29b1 admin admin] PATH_INFO: `/v3/projects/04a66f7a7ec84974ae65e72e6f7ecb43/users/670cbd3e80a44d00b69504b98573fcb9/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:55.395891 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab20fdf-a24c-4da7-90a8-22f57fee29b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04a66f7a7ec84974ae65e72e6f7ecb43, user_id=670cbd3e80a44d00b69504b98573fcb9, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:55.398545 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab20fdf-a24c-4da7-90a8-22f57fee29b1 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:55.409301 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1391/2738] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:37:55 2022] PUT /identity/v3/projects/04a66f7a7ec84974ae65e72e6f7ecb43/users/670cbd3e80a44d00b69504b98573fcb9/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:37:55.419983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72db591-a9d5-4b02-960e-d710981a37c5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:55.420279 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72db591-a9d5-4b02-960e-d710981a37c5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:55.420566 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72db591-a9d5-4b02-960e-d710981a37c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:55.446928 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d72db591-a9d5-4b02-960e-d710981a37c5 None None] MFA Rules not processed for user `670cbd3e80a44d00b69504b98573fcb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:37:55.448135 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d72db591-a9d5-4b02-960e-d710981a37c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:55.518438 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1348/2739] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:55 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:37:55.531157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab20fdf-a24c-4da7-90a8-22f57fee29b1 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:55.586639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab20fdf-a24c-4da7-90a8-22f57fee29b1 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:55.591004 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105a7ac-d43d-4c39-8600-37028cd9657e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:55.591647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105a7ac-d43d-4c39-8600-37028cd9657e service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:55.591927 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105a7ac-d43d-4c39-8600-37028cd9657e service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:55.592217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105a7ac-d43d-4c39-8600-37028cd9657e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:55.601047 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105a7ac-d43d-4c39-8600-37028cd9657e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:55.602498 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105a7ac-d43d-4c39-8600-37028cd9657e service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:55.611968 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1392/2740] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:55 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:55.685551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72db591-a9d5-4b02-960e-d710981a37c5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:55.695012 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72db591-a9d5-4b02-960e-d710981a37c5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:55.698658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895eaa5-0063-4e72-88bc-06c0cae231fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:55.699299 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895eaa5-0063-4e72-88bc-06c0cae231fb service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:55.700298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895eaa5-0063-4e72-88bc-06c0cae231fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:55.700298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895eaa5-0063-4e72-88bc-06c0cae231fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:55.757584 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895eaa5-0063-4e72-88bc-06c0cae231fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:55.759079 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895eaa5-0063-4e72-88bc-06c0cae231fb service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:55.769055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1349/2741] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:56.325992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105a7ac-d43d-4c39-8600-37028cd9657e service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:56.327416 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d105a7ac-d43d-4c39-8600-37028cd9657e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:56.338438 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105a7ac-d43d-4c39-8600-37028cd9657e service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:56.342328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418be2ac-af24-4493-84b4-26c91f21ee99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:56.342956 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418be2ac-af24-4493-84b4-26c91f21ee99 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:56.343237 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418be2ac-af24-4493-84b4-26c91f21ee99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:56.343507 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418be2ac-af24-4493-84b4-26c91f21ee99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:56.345341 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-418be2ac-af24-4493-84b4-26c91f21ee99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:37:56.404954 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418be2ac-af24-4493-84b4-26c91f21ee99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:56.406411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418be2ac-af24-4493-84b4-26c91f21ee99 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:56.415080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1393/2742] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:37:56.924110 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895eaa5-0063-4e72-88bc-06c0cae231fb service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:56.932697 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895eaa5-0063-4e72-88bc-06c0cae231fb service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:56.936320 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4e464-8be3-4a06-8049-2344c12cd4bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:56.936963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4e464-8be3-4a06-8049-2344c12cd4bf service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:56.937284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4e464-8be3-4a06-8049-2344c12cd4bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:56.937560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4e464-8be3-4a06-8049-2344c12cd4bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:56.997929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4e464-8be3-4a06-8049-2344c12cd4bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:56.999370 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4e464-8be3-4a06-8049-2344c12cd4bf service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:57.009160 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1350/2743] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:37:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:37:59.825834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418be2ac-af24-4493-84b4-26c91f21ee99 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:37:59.888995 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418be2ac-af24-4493-84b4-26c91f21ee99 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:37:59.893938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265119fc-6f7f-4c56-b08e-6382cbb4a7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:37:59.894925 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265119fc-6f7f-4c56-b08e-6382cbb4a7f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:59.895309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265119fc-6f7f-4c56-b08e-6382cbb4a7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:59.895585 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265119fc-6f7f-4c56-b08e-6382cbb4a7f3 admin admin] PATH_INFO: `/v3/users/f5898343f15142ad96152fac7d86bbe2` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:37:59.896835 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265119fc-6f7f-4c56-b08e-6382cbb4a7f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5898343f15142ad96152fac7d86bbe2)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:37:59.899009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265119fc-6f7f-4c56-b08e-6382cbb4a7f3 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:37:59.928818 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:37:59.950129 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-265119fc-6f7f-4c56-b08e-6382cbb4a7f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5898343f15142ad96152fac7d86bbe2'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:59.961651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-265119fc-6f7f-4c56-b08e-6382cbb4a7f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5898343f15142ad96152fac7d86bbe2'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:59.962307 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-265119fc-6f7f-4c56-b08e-6382cbb4a7f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5898343f15142ad96152fac7d86bbe2'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:59.969581 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-265119fc-6f7f-4c56-b08e-6382cbb4a7f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5898343f15142ad96152fac7d86bbe2'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:59.970179 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-265119fc-6f7f-4c56-b08e-6382cbb4a7f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5898343f15142ad96152fac7d86bbe2'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:37:59.982920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1394/2744] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:37:59 2022] DELETE /identity/v3/users/f5898343f15142ad96152fac7d86bbe2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:37:59.987618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd4a40-ca29-434f-b276-e08cab0b72c5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:37:59.988106 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd4a40-ca29-434f-b276-e08cab0b72c5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:37:59.988216 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd4a40-ca29-434f-b276-e08cab0b72c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:00.005885 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-fdcd4a40-ca29-434f-b276-e08cab0b72c5 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:00.007097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fdcd4a40-ca29-434f-b276-e08cab0b72c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:00.016350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:00.068007 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1351/2745] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:00.079009 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265119fc-6f7f-4c56-b08e-6382cbb4a7f3 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:00.080908 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-265119fc-6f7f-4c56-b08e-6382cbb4a7f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:00.141843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265119fc-6f7f-4c56-b08e-6382cbb4a7f3 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:00.147147 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147eddb-2f9e-4479-8600-9ba55da2eb97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:00.147858 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147eddb-2f9e-4479-8600-9ba55da2eb97 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:00.148143 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147eddb-2f9e-4479-8600-9ba55da2eb97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:00.149079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147eddb-2f9e-4479-8600-9ba55da2eb97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:00.158768 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147eddb-2f9e-4479-8600-9ba55da2eb97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:00.160217 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147eddb-2f9e-4479-8600-9ba55da2eb97 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:00.170542 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1395/2746] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:38:00.446023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd4a40-ca29-434f-b276-e08cab0b72c5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:00.447395 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fdcd4a40-ca29-434f-b276-e08cab0b72c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:00.460158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd4a40-ca29-434f-b276-e08cab0b72c5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:00.464719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25511af-d3ff-4977-9cc3-da0b5679faac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:00.466703 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25511af-d3ff-4977-9cc3-da0b5679faac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:00.466993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25511af-d3ff-4977-9cc3-da0b5679faac admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:00.467270 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25511af-d3ff-4977-9cc3-da0b5679faac admin admin] PATH_INFO: `/v3/projects/e5341f7e995b4536abfc75d5e3295ffc` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:00.468486 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25511af-d3ff-4977-9cc3-da0b5679faac admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5341f7e995b4536abfc75d5e3295ffc)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:00.470432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25511af-d3ff-4977-9cc3-da0b5679faac admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:00.518173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a25511af-d3ff-4977-9cc3-da0b5679faac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5341f7e995b4536abfc75d5e3295ffc', 'request_id': 'req-a25511af-d3ff-4977-9cc3-da0b5679faac'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:00.525363 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a25511af-d3ff-4977-9cc3-da0b5679faac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5341f7e995b4536abfc75d5e3295ffc', 'request_id': 'req-a25511af-d3ff-4977-9cc3-da0b5679faac'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:00.532209 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a25511af-d3ff-4977-9cc3-da0b5679faac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5341f7e995b4536abfc75d5e3295ffc', 'request_id': 'req-a25511af-d3ff-4977-9cc3-da0b5679faac'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:00.544325 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a25511af-d3ff-4977-9cc3-da0b5679faac admin admin] The token cache is being invalidate because project e5341f7e995b4536abfc75d5e3295ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:38:00.544617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a25511af-d3ff-4977-9cc3-da0b5679faac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:00.545305 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a25511af-d3ff-4977-9cc3-da0b5679faac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:00.547146 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1352/2747] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:38:00 2022] DELETE /identity/v3/projects/e5341f7e995b4536abfc75d5e3295ffc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:00.551145 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147eddb-2f9e-4479-8600-9ba55da2eb97 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:00.560931 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147eddb-2f9e-4479-8600-9ba55da2eb97 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:00.615300 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea27c43-9329-4410-89e0-224b87f5b50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:00.616190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea27c43-9329-4410-89e0-224b87f5b50e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:00.616472 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea27c43-9329-4410-89e0-224b87f5b50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:00.616747 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea27c43-9329-4410-89e0-224b87f5b50e admin admin] PATH_INFO: `/v3/users/670cbd3e80a44d00b69504b98573fcb9` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:00.617945 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea27c43-9329-4410-89e0-224b87f5b50e admin admin] RBAC: Authorizing `identity:delete_user(user_id=670cbd3e80a44d00b69504b98573fcb9)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:00.619856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea27c43-9329-4410-89e0-224b87f5b50e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:00.666195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-bea27c43-9329-4410-89e0-224b87f5b50e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '670cbd3e80a44d00b69504b98573fcb9'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:00.667294 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:00.681419 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-bea27c43-9329-4410-89e0-224b87f5b50e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '670cbd3e80a44d00b69504b98573fcb9'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:00.682072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-bea27c43-9329-4410-89e0-224b87f5b50e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '670cbd3e80a44d00b69504b98573fcb9'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:00.689462 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-bea27c43-9329-4410-89e0-224b87f5b50e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '670cbd3e80a44d00b69504b98573fcb9'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:00.690067 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-bea27c43-9329-4410-89e0-224b87f5b50e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '670cbd3e80a44d00b69504b98573fcb9'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:00.703129 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1396/2748] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:38:00 2022] DELETE /identity/v3/users/670cbd3e80a44d00b69504b98573fcb9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:00.858799 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25511af-d3ff-4977-9cc3-da0b5679faac admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:00.860594 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a25511af-d3ff-4977-9cc3-da0b5679faac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:00.874525 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25511af-d3ff-4977-9cc3-da0b5679faac admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:00.935260 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575c28e-a956-413a-8d9c-e6b22762ee22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:00.936149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575c28e-a956-413a-8d9c-e6b22762ee22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:00.936559 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575c28e-a956-413a-8d9c-e6b22762ee22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:00.936848 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575c28e-a956-413a-8d9c-e6b22762ee22 admin admin] PATH_INFO: `/v3/users/1cadf8f5d109489189c0d4e299f57d29` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:00.938080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575c28e-a956-413a-8d9c-e6b22762ee22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cadf8f5d109489189c0d4e299f57d29)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:00.940020 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575c28e-a956-413a-8d9c-e6b22762ee22 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:00.984821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:00.991475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea27c43-9329-4410-89e0-224b87f5b50e admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:00.992921 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2575c28e-a956-413a-8d9c-e6b22762ee22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cadf8f5d109489189c0d4e299f57d29'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:00.994961 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-bea27c43-9329-4410-89e0-224b87f5b50e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:01.006780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2575c28e-a956-413a-8d9c-e6b22762ee22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cadf8f5d109489189c0d4e299f57d29'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.007508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2575c28e-a956-413a-8d9c-e6b22762ee22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cadf8f5d109489189c0d4e299f57d29'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.008121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2575c28e-a956-413a-8d9c-e6b22762ee22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cadf8f5d109489189c0d4e299f57d29'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.016294 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2575c28e-a956-413a-8d9c-e6b22762ee22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cadf8f5d109489189c0d4e299f57d29'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.023666 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea27c43-9329-4410-89e0-224b87f5b50e admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:01.029207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a2814-46e0-4e63-96cf-64d4db432c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:01.030937 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a2814-46e0-4e63-96cf-64d4db432c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:01.030937 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a2814-46e0-4e63-96cf-64d4db432c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:01.030937 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a2814-46e0-4e63-96cf-64d4db432c45 admin admin] PATH_INFO: `/v3/projects/04a66f7a7ec84974ae65e72e6f7ecb43` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:01.031193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1353/2749] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:38:00 2022] DELETE /identity/v3/users/1cadf8f5d109489189c0d4e299f57d29 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:01.032243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a2814-46e0-4e63-96cf-64d4db432c45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04a66f7a7ec84974ae65e72e6f7ecb43)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:01.034537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a2814-46e0-4e63-96cf-64d4db432c45 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:01.035414 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe3855-ab7d-4e68-bae6-7d6b20a4ccfd None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:01.035714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe3855-ab7d-4e68-bae6-7d6b20a4ccfd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:01.035995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe3855-ab7d-4e68-bae6-7d6b20a4ccfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:01.053453 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3dbe3855-ab7d-4e68-bae6-7d6b20a4ccfd None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:01.054672 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3dbe3855-ab7d-4e68-bae6-7d6b20a4ccfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:01.090423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-282a2814-46e0-4e63-96cf-64d4db432c45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04a66f7a7ec84974ae65e72e6f7ecb43', 'request_id': 'req-282a2814-46e0-4e63-96cf-64d4db432c45'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.096132 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-282a2814-46e0-4e63-96cf-64d4db432c45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04a66f7a7ec84974ae65e72e6f7ecb43', 'request_id': 'req-282a2814-46e0-4e63-96cf-64d4db432c45'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.102497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-282a2814-46e0-4e63-96cf-64d4db432c45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04a66f7a7ec84974ae65e72e6f7ecb43', 'request_id': 'req-282a2814-46e0-4e63-96cf-64d4db432c45'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.116482 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-282a2814-46e0-4e63-96cf-64d4db432c45 admin admin] The token cache is being invalidate because project 04a66f7a7ec84974ae65e72e6f7ecb43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:38:01.116779 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-282a2814-46e0-4e63-96cf-64d4db432c45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.117414 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-282a2814-46e0-4e63-96cf-64d4db432c45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.118942 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1397/2750] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:38:00 2022] DELETE /identity/v3/projects/04a66f7a7ec84974ae65e72e6f7ecb43 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:01.122243 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1354/2751] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:38:01.124202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98bd01-c744-4a65-b551-44d6b87d072f None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:01.124495 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98bd01-c744-4a65-b551-44d6b87d072f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:01.124779 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98bd01-c744-4a65-b551-44d6b87d072f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:01.131765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe3855-ab7d-4e68-bae6-7d6b20a4ccfd None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:01.133131 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3dbe3855-ab7d-4e68-bae6-7d6b20a4ccfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:01.144923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-eb98bd01-c744-4a65-b551-44d6b87d072f None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:01.146159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-eb98bd01-c744-4a65-b551-44d6b87d072f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:01.192483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe3855-ab7d-4e68-bae6-7d6b20a4ccfd None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:01.207617 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1398/2752] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:01.215267 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411dadd5-bd9d-45d7-a5b3-251de2e5da76 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:01.215565 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411dadd5-bd9d-45d7-a5b3-251de2e5da76 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:01.215838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411dadd5-bd9d-45d7-a5b3-251de2e5da76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:01.233849 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-411dadd5-bd9d-45d7-a5b3-251de2e5da76 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:01.235281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-411dadd5-bd9d-45d7-a5b3-251de2e5da76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:01.243325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1399/2753] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:38:01.243591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98668c-ee62-48bf-9640-8ee88f42205c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:01.244008 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98668c-ee62-48bf-9640-8ee88f42205c service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:01.244307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98668c-ee62-48bf-9640-8ee88f42205c service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:01.244650 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98668c-ee62-48bf-9640-8ee88f42205c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:01.247160 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-be98668c-ee62-48bf-9640-8ee88f42205c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:01.248609 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411dadd5-bd9d-45d7-a5b3-251de2e5da76 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:01.258316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411dadd5-bd9d-45d7-a5b3-251de2e5da76 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:01.259523 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98668c-ee62-48bf-9640-8ee88f42205c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:01.260961 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98668c-ee62-48bf-9640-8ee88f42205c service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:01.264234 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156439a-9ec6-4a4a-bddc-e493a02860f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:01.264984 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156439a-9ec6-4a4a-bddc-e493a02860f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:01.265363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156439a-9ec6-4a4a-bddc-e493a02860f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:01.265640 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156439a-9ec6-4a4a-bddc-e493a02860f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:01.266446 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156439a-9ec6-4a4a-bddc-e493a02860f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:38:01.266753 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156439a-9ec6-4a4a-bddc-e493a02860f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:01.268613 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156439a-9ec6-4a4a-bddc-e493a02860f7 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:01.272081 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1355/2754] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:38:01.283867 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1400/2755] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:38:01.284964 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98668c-ee62-48bf-9640-8ee88f42205c service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:01.295240 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98668c-ee62-48bf-9640-8ee88f42205c service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:01.301504 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c28d55-be9c-47a9-9fb1-62e42f2ae4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:01.302625 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c28d55-be9c-47a9-9fb1-62e42f2ae4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:01.302916 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c28d55-be9c-47a9-9fb1-62e42f2ae4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:01.303277 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c28d55-be9c-47a9-9fb1-62e42f2ae4c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:01.303990 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c28d55-be9c-47a9-9fb1-62e42f2ae4c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:01.306514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c28d55-be9c-47a9-9fb1-62e42f2ae4c6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:01.323150 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:01.340804 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1356/2756] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:38:01 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:38:01.342337 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156439a-9ec6-4a4a-bddc-e493a02860f7 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:01.353029 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156439a-9ec6-4a4a-bddc-e493a02860f7 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:01.380999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:01.419901 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f63be-5cb3-46b7-b40b-c06ee0ae365b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:01.420662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f63be-5cb3-46b7-b40b-c06ee0ae365b admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:01.420986 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f63be-5cb3-46b7-b40b-c06ee0ae365b admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:01.421309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f63be-5cb3-46b7-b40b-c06ee0ae365b admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:01.422088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f63be-5cb3-46b7-b40b-c06ee0ae365b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:01.424137 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f63be-5cb3-46b7-b40b-c06ee0ae365b admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:01.446579 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1401/2757] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:38:01 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:38:01.450999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c28d55-be9c-47a9-9fb1-62e42f2ae4c6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:01.461568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c28d55-be9c-47a9-9fb1-62e42f2ae4c6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:01.467201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd3adf-a987-4a34-ba79-05adab637547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:01.467943 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd3adf-a987-4a34-ba79-05adab637547 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:01.468306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd3adf-a987-4a34-ba79-05adab637547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:01.468656 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd3adf-a987-4a34-ba79-05adab637547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:01.469495 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd3adf-a987-4a34-ba79-05adab637547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:01.471171 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd3adf-a987-4a34-ba79-05adab637547 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:01.486681 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1357/2758] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:38:01.487857 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f63be-5cb3-46b7-b40b-c06ee0ae365b admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:01.498165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f63be-5cb3-46b7-b40b-c06ee0ae365b admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:01.502922 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7159b2d-1d67-4bb8-862c-3dbe4f4fae43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:01.503597 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7159b2d-1d67-4bb8-862c-3dbe4f4fae43 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:01.503884 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7159b2d-1d67-4bb8-862c-3dbe4f4fae43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:01.504165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7159b2d-1d67-4bb8-862c-3dbe4f4fae43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:01.504965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7159b2d-1d67-4bb8-862c-3dbe4f4fae43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:01.506633 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7159b2d-1d67-4bb8-862c-3dbe4f4fae43 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:01.517793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1402/2759] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:38:01.521943 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd3adf-a987-4a34-ba79-05adab637547 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:01.532923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd3adf-a987-4a34-ba79-05adab637547 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:01.537746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43d03f-b974-404a-b788-ad0ff177849d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:01.538645 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43d03f-b974-404a-b788-ad0ff177849d admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:01.538911 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43d03f-b974-404a-b788-ad0ff177849d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:01.539177 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43d03f-b974-404a-b788-ad0ff177849d admin admin] PATH_INFO: `/v3/projects/c195c8ef7e4842808e8d512284db3ed5/users/97dd40e0fa1d4ec08bd30857af9031da/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:01.555537 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7159b2d-1d67-4bb8-862c-3dbe4f4fae43 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:01.557610 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43d03f-b974-404a-b788-ad0ff177849d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c195c8ef7e4842808e8d512284db3ed5, user_id=97dd40e0fa1d4ec08bd30857af9031da, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:01.558052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c7159b2d-1d67-4bb8-862c-3dbe4f4fae43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:01.561358 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43d03f-b974-404a-b788-ad0ff177849d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:01.576716 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7159b2d-1d67-4bb8-862c-3dbe4f4fae43 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:01.579102 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1358/2760] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:38:01 2022] PUT /identity/v3/projects/c195c8ef7e4842808e8d512284db3ed5/users/97dd40e0fa1d4ec08bd30857af9031da/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:01.589172 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f63d3a-7857-420f-b640-ab3b9be1a0b3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:01.589665 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f63d3a-7857-420f-b640-ab3b9be1a0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:01.590535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f63d3a-7857-420f-b640-ab3b9be1a0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:01.617309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-29f63d3a-7857-420f-b640-ab3b9be1a0b3 None None] MFA Rules not processed for user `97dd40e0fa1d4ec08bd30857af9031da`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:01.618457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-29f63d3a-7857-420f-b640-ab3b9be1a0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:01.648308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:01.649320 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:01.649752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:01.650040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd admin admin] PATH_INFO: `/v3/projects/6f64c59d39b9480bab8f007fbb1dac5f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:01.651283 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f64c59d39b9480bab8f007fbb1dac5f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:01.654156 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:01.686490 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1359/2761] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:38:01.696968 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f63d3a-7857-420f-b640-ab3b9be1a0b3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:01.698580 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-29f63d3a-7857-420f-b640-ab3b9be1a0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:01.701870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f64c59d39b9480bab8f007fbb1dac5f', 'request_id': 'req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.707720 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f64c59d39b9480bab8f007fbb1dac5f', 'request_id': 'req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.714048 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f64c59d39b9480bab8f007fbb1dac5f', 'request_id': 'req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.726887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd admin admin] The token cache is being invalidate because project 6f64c59d39b9480bab8f007fbb1dac5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:38:01.727192 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.727813 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.729399 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1403/2762] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:38:01 2022] DELETE /identity/v3/projects/6f64c59d39b9480bab8f007fbb1dac5f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:01.733689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:01.735107 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:01.763064 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f63d3a-7857-420f-b640-ab3b9be1a0b3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:01.778928 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:01.802483 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4359cb-48ef-45fd-ae49-3edbd2ae20bd admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:01.807159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8238a-cd68-4c02-b682-809645a3e2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:01.808032 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8238a-cd68-4c02-b682-809645a3e2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:01.808316 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8238a-cd68-4c02-b682-809645a3e2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:01.809192 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8238a-cd68-4c02-b682-809645a3e2ed admin admin] PATH_INFO: `/v3/users/2ccc27d2267e437d8f1c15a59a6ba65b` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:01.810411 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8238a-cd68-4c02-b682-809645a3e2ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ccc27d2267e437d8f1c15a59a6ba65b)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:01.812347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8238a-cd68-4c02-b682-809645a3e2ed admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:01.813529 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e279f53-0f5a-46e2-86a1-e91422c889de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:01.814236 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e279f53-0f5a-46e2-86a1-e91422c889de service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:01.814524 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e279f53-0f5a-46e2-86a1-e91422c889de service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:01.814800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e279f53-0f5a-46e2-86a1-e91422c889de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:01.816617 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2e279f53-0f5a-46e2-86a1-e91422c889de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:01.837443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:01.862030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-25a8238a-cd68-4c02-b682-809645a3e2ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ccc27d2267e437d8f1c15a59a6ba65b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.874679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-25a8238a-cd68-4c02-b682-809645a3e2ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ccc27d2267e437d8f1c15a59a6ba65b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.875304 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-25a8238a-cd68-4c02-b682-809645a3e2ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ccc27d2267e437d8f1c15a59a6ba65b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.882914 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e279f53-0f5a-46e2-86a1-e91422c889de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:01.883100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-25a8238a-cd68-4c02-b682-809645a3e2ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ccc27d2267e437d8f1c15a59a6ba65b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.883786 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-25a8238a-cd68-4c02-b682-809645a3e2ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ccc27d2267e437d8f1c15a59a6ba65b'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:01.884670 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e279f53-0f5a-46e2-86a1-e91422c889de service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:01.894887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1360/2763] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:38:01.897615 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1404/2764] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:38:01 2022] DELETE /identity/v3/users/2ccc27d2267e437d8f1c15a59a6ba65b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:38:02.191391 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e279f53-0f5a-46e2-86a1-e91422c889de service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:02.206737 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e279f53-0f5a-46e2-86a1-e91422c889de service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:02.283840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3f617-56cf-4ccb-9841-0655eecd43e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:02.284727 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3f617-56cf-4ccb-9841-0655eecd43e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:02.285085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3f617-56cf-4ccb-9841-0655eecd43e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:02.285593 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3f617-56cf-4ccb-9841-0655eecd43e1 admin admin] PATH_INFO: `/v3/projects/e168c6c648b746c3a9bef7219a6ddd2d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:02.287032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3f617-56cf-4ccb-9841-0655eecd43e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e168c6c648b746c3a9bef7219a6ddd2d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:02.290021 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3f617-56cf-4ccb-9841-0655eecd43e1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:02.351751 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-53a3f617-56cf-4ccb-9841-0655eecd43e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e168c6c648b746c3a9bef7219a6ddd2d', 'request_id': 'req-53a3f617-56cf-4ccb-9841-0655eecd43e1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:02.365662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-53a3f617-56cf-4ccb-9841-0655eecd43e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e168c6c648b746c3a9bef7219a6ddd2d', 'request_id': 'req-53a3f617-56cf-4ccb-9841-0655eecd43e1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:02.372724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-53a3f617-56cf-4ccb-9841-0655eecd43e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e168c6c648b746c3a9bef7219a6ddd2d', 'request_id': 'req-53a3f617-56cf-4ccb-9841-0655eecd43e1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:02.385477 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-53a3f617-56cf-4ccb-9841-0655eecd43e1 admin admin] The token cache is being invalidate because project e168c6c648b746c3a9bef7219a6ddd2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:38:02.385987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-53a3f617-56cf-4ccb-9841-0655eecd43e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:02.387727 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-53a3f617-56cf-4ccb-9841-0655eecd43e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:02.389711 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1361/2765] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:38:02 2022] DELETE /identity/v3/projects/e168c6c648b746c3a9bef7219a6ddd2d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:02.394048 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8238a-cd68-4c02-b682-809645a3e2ed admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:02.395555 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-25a8238a-cd68-4c02-b682-809645a3e2ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:02.409653 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8238a-cd68-4c02-b682-809645a3e2ed admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:02.469347 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7a79a-ed0d-4610-8610-5c3bb6db81c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:02.470233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7a79a-ed0d-4610-8610-5c3bb6db81c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:02.470516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7a79a-ed0d-4610-8610-5c3bb6db81c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:02.470800 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7a79a-ed0d-4610-8610-5c3bb6db81c3 admin admin] PATH_INFO: `/v3/users/fec132f9fbe64cdf804061a9f9e39a8e` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:02.472004 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7a79a-ed0d-4610-8610-5c3bb6db81c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fec132f9fbe64cdf804061a9f9e39a8e)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:02.473992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7a79a-ed0d-4610-8610-5c3bb6db81c3 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:02.522839 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a2b7a79a-ed0d-4610-8610-5c3bb6db81c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fec132f9fbe64cdf804061a9f9e39a8e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:02.531035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:02.536369 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a2b7a79a-ed0d-4610-8610-5c3bb6db81c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fec132f9fbe64cdf804061a9f9e39a8e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:02.537241 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a2b7a79a-ed0d-4610-8610-5c3bb6db81c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fec132f9fbe64cdf804061a9f9e39a8e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:02.544782 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a2b7a79a-ed0d-4610-8610-5c3bb6db81c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fec132f9fbe64cdf804061a9f9e39a8e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:02.545475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-a2b7a79a-ed0d-4610-8610-5c3bb6db81c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fec132f9fbe64cdf804061a9f9e39a8e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:02.557804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1405/2766] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:38:02 2022] DELETE /identity/v3/users/fec132f9fbe64cdf804061a9f9e39a8e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:02.830465 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3f617-56cf-4ccb-9841-0655eecd43e1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:02.831992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-53a3f617-56cf-4ccb-9841-0655eecd43e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:02.845008 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3f617-56cf-4ccb-9841-0655eecd43e1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:02.897618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:02.908813 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358864e-6830-4fde-932b-73263de63fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:02.909647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358864e-6830-4fde-932b-73263de63fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:02.909934 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358864e-6830-4fde-932b-73263de63fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:02.910214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358864e-6830-4fde-932b-73263de63fd0 admin admin] PATH_INFO: `/v3/projects/9e986ceffc2a47e28052eba6a20af6f4` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:02.911441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358864e-6830-4fde-932b-73263de63fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e986ceffc2a47e28052eba6a20af6f4)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:02.913404 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358864e-6830-4fde-932b-73263de63fd0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:02.966624 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5358864e-6830-4fde-932b-73263de63fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e986ceffc2a47e28052eba6a20af6f4', 'request_id': 'req-5358864e-6830-4fde-932b-73263de63fd0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:02.973744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5358864e-6830-4fde-932b-73263de63fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e986ceffc2a47e28052eba6a20af6f4', 'request_id': 'req-5358864e-6830-4fde-932b-73263de63fd0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:02.979512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5358864e-6830-4fde-932b-73263de63fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e986ceffc2a47e28052eba6a20af6f4', 'request_id': 'req-5358864e-6830-4fde-932b-73263de63fd0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:02.991420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5358864e-6830-4fde-932b-73263de63fd0 admin admin] The token cache is being invalidate because project 9e986ceffc2a47e28052eba6a20af6f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:38:02.991705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5358864e-6830-4fde-932b-73263de63fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:02.992320 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5358864e-6830-4fde-932b-73263de63fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:02.994055 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1362/2767] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:38:02 2022] DELETE /identity/v3/projects/9e986ceffc2a47e28052eba6a20af6f4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:02.999911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78bce4-dfbb-4ae8-80d8-218b4e40f61f None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:03.000225 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78bce4-dfbb-4ae8-80d8-218b4e40f61f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:03.000504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78bce4-dfbb-4ae8-80d8-218b4e40f61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:03.018815 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-ba78bce4-dfbb-4ae8-80d8-218b4e40f61f None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:03.020161 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ba78bce4-dfbb-4ae8-80d8-218b4e40f61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:03.081758 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1406/2768] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:03.089587 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27890b0e-29d5-471a-a8df-aea2ad93e15a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:03.090440 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27890b0e-29d5-471a-a8df-aea2ad93e15a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:03.090731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27890b0e-29d5-471a-a8df-aea2ad93e15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:03.109618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-27890b0e-29d5-471a-a8df-aea2ad93e15a None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:03.110840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-27890b0e-29d5-471a-a8df-aea2ad93e15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:03.120420 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1363/2769] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:03.125748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78bce4-dfbb-4ae8-80d8-218b4e40f61f None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:03.135356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78bce4-dfbb-4ae8-80d8-218b4e40f61f None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:03.139952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac981e4-166d-41f3-9a27-d104328b61be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:03.140620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac981e4-166d-41f3-9a27-d104328b61be admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:03.140899 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac981e4-166d-41f3-9a27-d104328b61be admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:03.141198 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac981e4-166d-41f3-9a27-d104328b61be admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:03.142218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac981e4-166d-41f3-9a27-d104328b61be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:38:03.142524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac981e4-166d-41f3-9a27-d104328b61be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:03.144116 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac981e4-166d-41f3-9a27-d104328b61be admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:03.155977 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1407/2770] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:38:03.160117 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27890b0e-29d5-471a-a8df-aea2ad93e15a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:03.170580 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27890b0e-29d5-471a-a8df-aea2ad93e15a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:03.175203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49999045-2afd-4534-ba31-7c5e8a7d53f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:03.175852 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49999045-2afd-4534-ba31-7c5e8a7d53f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:03.176144 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49999045-2afd-4534-ba31-7c5e8a7d53f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:03.176421 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49999045-2afd-4534-ba31-7c5e8a7d53f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:03.177107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49999045-2afd-4534-ba31-7c5e8a7d53f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:03.179028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49999045-2afd-4534-ba31-7c5e8a7d53f0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:03.205602 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1364/2771] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:38:03 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:38:03.209656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac981e4-166d-41f3-9a27-d104328b61be admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:03.220834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac981e4-166d-41f3-9a27-d104328b61be admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:03.281017 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c24638-21b0-46b2-ad0a-d92616661628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:03.281717 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c24638-21b0-46b2-ad0a-d92616661628 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:03.281999 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c24638-21b0-46b2-ad0a-d92616661628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:03.282268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c24638-21b0-46b2-ad0a-d92616661628 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:03.282947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c24638-21b0-46b2-ad0a-d92616661628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:03.284854 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c24638-21b0-46b2-ad0a-d92616661628 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:03.304614 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1408/2772] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:38:03 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:38:03.308727 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49999045-2afd-4534-ba31-7c5e8a7d53f0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:03.319766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49999045-2afd-4534-ba31-7c5e8a7d53f0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:03.324575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563ef9d-a01b-4e8a-a4b7-7863024894be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:03.325455 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563ef9d-a01b-4e8a-a4b7-7863024894be admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:03.325740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563ef9d-a01b-4e8a-a4b7-7863024894be admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:03.326023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563ef9d-a01b-4e8a-a4b7-7863024894be admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:03.326759 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563ef9d-a01b-4e8a-a4b7-7863024894be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:03.328399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563ef9d-a01b-4e8a-a4b7-7863024894be admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:03.343185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1365/2773] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:38:03.344396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c24638-21b0-46b2-ad0a-d92616661628 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:03.355176 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c24638-21b0-46b2-ad0a-d92616661628 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:03.360703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4f75f-cae6-4ad3-9138-b2709f74a499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:03.361516 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4f75f-cae6-4ad3-9138-b2709f74a499 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:03.362657 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4f75f-cae6-4ad3-9138-b2709f74a499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:03.363098 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4f75f-cae6-4ad3-9138-b2709f74a499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:03.363924 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4f75f-cae6-4ad3-9138-b2709f74a499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:03.365644 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4f75f-cae6-4ad3-9138-b2709f74a499 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:03.376767 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1409/2774] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:38:03.380828 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563ef9d-a01b-4e8a-a4b7-7863024894be admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:03.390995 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563ef9d-a01b-4e8a-a4b7-7863024894be admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:03.395902 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e0996-27e4-4c31-bcef-b7ddedb88109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:03.396794 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e0996-27e4-4c31-bcef-b7ddedb88109 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:03.397080 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e0996-27e4-4c31-bcef-b7ddedb88109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:03.397391 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e0996-27e4-4c31-bcef-b7ddedb88109 admin admin] PATH_INFO: `/v3/projects/5789bbb2be434abd89ed8c8f3842945a/users/c576d56b32f24e78a4f717df1e0bbd3e/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:03.416977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e0996-27e4-4c31-bcef-b7ddedb88109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5789bbb2be434abd89ed8c8f3842945a, user_id=c576d56b32f24e78a4f717df1e0bbd3e, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:03.420391 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e0996-27e4-4c31-bcef-b7ddedb88109 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:03.435222 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1366/2775] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:38:03 2022] PUT /identity/v3/projects/5789bbb2be434abd89ed8c8f3842945a/users/c576d56b32f24e78a4f717df1e0bbd3e/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:03.445860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2216-3eba-4b77-a9bb-c0affb7e2b79 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:03.446334 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2216-3eba-4b77-a9bb-c0affb7e2b79 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:03.446595 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2216-3eba-4b77-a9bb-c0affb7e2b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:03.474986 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-6b7f2216-3eba-4b77-a9bb-c0affb7e2b79 None None] MFA Rules not processed for user `c576d56b32f24e78a4f717df1e0bbd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:03.476239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-6b7f2216-3eba-4b77-a9bb-c0affb7e2b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:03.507773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:03.548500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1410/2776] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:03 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:03.567610 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e0996-27e4-4c31-bcef-b7ddedb88109 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:03.569477 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fa9e0996-27e4-4c31-bcef-b7ddedb88109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:03.632243 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e0996-27e4-4c31-bcef-b7ddedb88109 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:03.635880 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1905ab3-ac7a-4459-a6b4-d58f682104be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '215c6f6f1aad486f85ce20590d889cbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:03.637033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1905ab3-ac7a-4459-a6b4-d58f682104be service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:03.637557 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1905ab3-ac7a-4459-a6b4-d58f682104be service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:03.637815 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1905ab3-ac7a-4459-a6b4-d58f682104be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:03.649313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1905ab3-ac7a-4459-a6b4-d58f682104be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:03.650308 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1905ab3-ac7a-4459-a6b4-d58f682104be service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:03.661832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1367/2777] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:38:11.743812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f2216-3eba-4b77-a9bb-c0affb7e2b79 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:11.745264 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-6b7f2216-3eba-4b77-a9bb-c0affb7e2b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:11.810607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f2216-3eba-4b77-a9bb-c0affb7e2b79 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:11.815173 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1fe6a-c0b2-443e-a003-d53b0c5ace56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:11.816042 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1fe6a-c0b2-443e-a003-d53b0c5ace56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:11.816328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1fe6a-c0b2-443e-a003-d53b0c5ace56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:11.816607 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1fe6a-c0b2-443e-a003-d53b0c5ace56 admin admin] PATH_INFO: `/v3/users/97dd40e0fa1d4ec08bd30857af9031da` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:11.817810 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1fe6a-c0b2-443e-a003-d53b0c5ace56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97dd40e0fa1d4ec08bd30857af9031da)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:11.819732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1fe6a-c0b2-443e-a003-d53b0c5ace56 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:11.866405 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-bad1fe6a-c0b2-443e-a003-d53b0c5ace56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97dd40e0fa1d4ec08bd30857af9031da'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:11.878548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-bad1fe6a-c0b2-443e-a003-d53b0c5ace56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97dd40e0fa1d4ec08bd30857af9031da'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:11.879295 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-bad1fe6a-c0b2-443e-a003-d53b0c5ace56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97dd40e0fa1d4ec08bd30857af9031da'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:11.887265 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-bad1fe6a-c0b2-443e-a003-d53b0c5ace56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97dd40e0fa1d4ec08bd30857af9031da'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:11.888060 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-bad1fe6a-c0b2-443e-a003-d53b0c5ace56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97dd40e0fa1d4ec08bd30857af9031da'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:11.900368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1411/2778] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:38:11 2022] DELETE /identity/v3/users/97dd40e0fa1d4ec08bd30857af9031da => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:11.905756 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e717f9-79c1-4162-a2e6-fb5e51d7a490 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:11.906273 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e717f9-79c1-4162-a2e6-fb5e51d7a490 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:11.906384 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e717f9-79c1-4162-a2e6-fb5e51d7a490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:11.923262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-13e717f9-79c1-4162-a2e6-fb5e51d7a490 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:11.924473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-13e717f9-79c1-4162-a2e6-fb5e51d7a490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:11.940272 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:11.991010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1368/2779] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:12.000689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1fe6a-c0b2-443e-a003-d53b0c5ace56 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:12.002089 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-bad1fe6a-c0b2-443e-a003-d53b0c5ace56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:12.059380 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1fe6a-c0b2-443e-a003-d53b0c5ace56 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:12.063020 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d703e-dcd4-4615-8942-8025cd27bf16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:12.063659 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d703e-dcd4-4615-8942-8025cd27bf16 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:12.063940 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d703e-dcd4-4615-8942-8025cd27bf16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:12.064219 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d703e-dcd4-4615-8942-8025cd27bf16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:12.070086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:12.075936 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d703e-dcd4-4615-8942-8025cd27bf16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:12.078897 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d703e-dcd4-4615-8942-8025cd27bf16 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:12.090887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1412/2780] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:38:12.363272 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e717f9-79c1-4162-a2e6-fb5e51d7a490 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:12.364716 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-13e717f9-79c1-4162-a2e6-fb5e51d7a490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:12.377802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e717f9-79c1-4162-a2e6-fb5e51d7a490 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:12.382390 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111e7eb-4a7a-4820-8856-41f3230683f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:12.383257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111e7eb-4a7a-4820-8856-41f3230683f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:12.383542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111e7eb-4a7a-4820-8856-41f3230683f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:12.383820 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111e7eb-4a7a-4820-8856-41f3230683f7 admin admin] PATH_INFO: `/v3/projects/c195c8ef7e4842808e8d512284db3ed5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:12.385011 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111e7eb-4a7a-4820-8856-41f3230683f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c195c8ef7e4842808e8d512284db3ed5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:12.387590 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111e7eb-4a7a-4820-8856-41f3230683f7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:12.437851 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3111e7eb-4a7a-4820-8856-41f3230683f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c195c8ef7e4842808e8d512284db3ed5', 'request_id': 'req-3111e7eb-4a7a-4820-8856-41f3230683f7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:12.447200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3111e7eb-4a7a-4820-8856-41f3230683f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c195c8ef7e4842808e8d512284db3ed5', 'request_id': 'req-3111e7eb-4a7a-4820-8856-41f3230683f7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:12.455079 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3111e7eb-4a7a-4820-8856-41f3230683f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c195c8ef7e4842808e8d512284db3ed5', 'request_id': 'req-3111e7eb-4a7a-4820-8856-41f3230683f7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:12.467307 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3111e7eb-4a7a-4820-8856-41f3230683f7 admin admin] The token cache is being invalidate because project c195c8ef7e4842808e8d512284db3ed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:38:12.467735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3111e7eb-4a7a-4820-8856-41f3230683f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:12.468397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3111e7eb-4a7a-4820-8856-41f3230683f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:12.470178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1369/2781] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:38:12 2022] DELETE /identity/v3/projects/c195c8ef7e4842808e8d512284db3ed5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:12.475202 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae8ff5-8e64-49dd-b087-1c55a63a901b None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:12.475568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae8ff5-8e64-49dd-b087-1c55a63a901b None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:12.475844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae8ff5-8e64-49dd-b087-1c55a63a901b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:12.493990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-56ae8ff5-8e64-49dd-b087-1c55a63a901b None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:12.495170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-56ae8ff5-8e64-49dd-b087-1c55a63a901b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:12.553898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1413/2782] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:12.563089 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81556de0-4d36-4e03-a151-1e6220f14e34 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:12.563380 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81556de0-4d36-4e03-a151-1e6220f14e34 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:12.563659 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81556de0-4d36-4e03-a151-1e6220f14e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:12.640147 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-81556de0-4d36-4e03-a151-1e6220f14e34 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:12.641387 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-81556de0-4d36-4e03-a151-1e6220f14e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:12.661484 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1370/2783] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:12.670998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae8ff5-8e64-49dd-b087-1c55a63a901b None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:12.681544 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae8ff5-8e64-49dd-b087-1c55a63a901b None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:12.696882 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a23df3-9029-47f3-bc17-8695ea6424c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:12.697693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a23df3-9029-47f3-bc17-8695ea6424c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:12.698061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a23df3-9029-47f3-bc17-8695ea6424c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:12.698396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a23df3-9029-47f3-bc17-8695ea6424c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:12.699281 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a23df3-9029-47f3-bc17-8695ea6424c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:38:12.699598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a23df3-9029-47f3-bc17-8695ea6424c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:12.701298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a23df3-9029-47f3-bc17-8695ea6424c9 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:12.713809 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1414/2784] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:38:12.719008 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81556de0-4d36-4e03-a151-1e6220f14e34 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:12.762292 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:12.764373 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81556de0-4d36-4e03-a151-1e6220f14e34 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:12.770281 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1b4ac-9bb5-4bad-a77a-dac529af4be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:12.771092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1b4ac-9bb5-4bad-a77a-dac529af4be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:12.771585 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1b4ac-9bb5-4bad-a77a-dac529af4be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:12.771923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1b4ac-9bb5-4bad-a77a-dac529af4be4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:12.772811 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1b4ac-9bb5-4bad-a77a-dac529af4be4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:12.774769 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1b4ac-9bb5-4bad-a77a-dac529af4be4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:12.800485 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1371/2785] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:38:12 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:38:12.804157 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a23df3-9029-47f3-bc17-8695ea6424c9 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:12.840471 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a23df3-9029-47f3-bc17-8695ea6424c9 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:12.896063 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a47f0-5dad-4205-a785-fb5c8a8732f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:12.896732 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a47f0-5dad-4205-a785-fb5c8a8732f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:12.897018 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a47f0-5dad-4205-a785-fb5c8a8732f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:12.897333 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a47f0-5dad-4205-a785-fb5c8a8732f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:12.898011 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3a47f0-5dad-4205-a785-fb5c8a8732f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:12.899931 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3a47f0-5dad-4205-a785-fb5c8a8732f1 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:12.925737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1415/2786] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:38:12 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:38:12.929738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1b4ac-9bb5-4bad-a77a-dac529af4be4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:12.941605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1b4ac-9bb5-4bad-a77a-dac529af4be4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:12.946301 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13259e31-3372-4b10-b0be-91162f00066d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:12.946971 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13259e31-3372-4b10-b0be-91162f00066d admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:12.947257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13259e31-3372-4b10-b0be-91162f00066d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:12.947535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13259e31-3372-4b10-b0be-91162f00066d admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:12.948268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13259e31-3372-4b10-b0be-91162f00066d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:12.950394 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13259e31-3372-4b10-b0be-91162f00066d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:12.960336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1372/2787] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:38:12.965376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a47f0-5dad-4205-a785-fb5c8a8732f1 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:12.975613 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a47f0-5dad-4205-a785-fb5c8a8732f1 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:12.981172 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d1ae1-a798-455c-9513-9fb38aa525dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:12.981891 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d1ae1-a798-455c-9513-9fb38aa525dc admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:12.982204 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d1ae1-a798-455c-9513-9fb38aa525dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:12.982486 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d1ae1-a798-455c-9513-9fb38aa525dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:12.983225 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d1ae1-a798-455c-9513-9fb38aa525dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:12.984827 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d1ae1-a798-455c-9513-9fb38aa525dc admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:12.994758 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1416/2788] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:38:12.998997 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13259e31-3372-4b10-b0be-91162f00066d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:13.009492 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13259e31-3372-4b10-b0be-91162f00066d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:13.015649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b471b-29a6-4042-8f6a-3785c71372c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:13.016560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b471b-29a6-4042-8f6a-3785c71372c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:13.017120 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b471b-29a6-4042-8f6a-3785c71372c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:13.017413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b471b-29a6-4042-8f6a-3785c71372c0 admin admin] PATH_INFO: `/v3/projects/2e4fe057576e4bcba6647814498ca197/users/d63df11e4d4344f3b1f6434c4e4a050a/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:13.162680 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b471b-29a6-4042-8f6a-3785c71372c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e4fe057576e4bcba6647814498ca197, user_id=d63df11e4d4344f3b1f6434c4e4a050a, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:13.165481 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b471b-29a6-4042-8f6a-3785c71372c0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:13.180506 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1373/2789] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:38:12 2022] PUT /identity/v3/projects/2e4fe057576e4bcba6647814498ca197/users/d63df11e4d4344f3b1f6434c4e4a050a/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:13.190736 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de64e2f-2409-4b42-819a-439a4cdcd064 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:13.191035 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de64e2f-2409-4b42-819a-439a4cdcd064 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:13.191346 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de64e2f-2409-4b42-819a-439a4cdcd064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:13.218898 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-7de64e2f-2409-4b42-819a-439a4cdcd064 None None] MFA Rules not processed for user `d63df11e4d4344f3b1f6434c4e4a050a`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:13.220118 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-7de64e2f-2409-4b42-819a-439a4cdcd064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:13.233357 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:13.287349 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1417/2790] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:13.298782 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b471b-29a6-4042-8f6a-3785c71372c0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:13.300577 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bc2b471b-29a6-4042-8f6a-3785c71372c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:13.374856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b471b-29a6-4042-8f6a-3785c71372c0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:13.378519 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c20f5-9ed3-4be8-b9e8-fa242e35b4cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:13.379168 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c20f5-9ed3-4be8-b9e8-fa242e35b4cb service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:13.379451 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c20f5-9ed3-4be8-b9e8-fa242e35b4cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:13.379723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c20f5-9ed3-4be8-b9e8-fa242e35b4cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:13.389118 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c20f5-9ed3-4be8-b9e8-fa242e35b4cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:13.390558 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c20f5-9ed3-4be8-b9e8-fa242e35b4cb service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:13.401001 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1374/2791] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:38:14.754067 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15df878-f153-4692-b664-17a33aba8487 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:14.754622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15df878-f153-4692-b664-17a33aba8487 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:14.754622 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15df878-f153-4692-b664-17a33aba8487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:14.772789 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-e15df878-f153-4692-b664-17a33aba8487 None None] MFA Rules not processed for user `d63df11e4d4344f3b1f6434c4e4a050a`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:14.774032 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e15df878-f153-4692-b664-17a33aba8487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:14.782037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1418/2792] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:23.395936 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=55,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:24.767703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=23 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:25.745673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c20f5-9ed3-4be8-b9e8-fa242e35b4cb service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:25.747441 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c32c20f5-9ed3-4be8-b9e8-fa242e35b4cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:25.876510 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c20f5-9ed3-4be8-b9e8-fa242e35b4cb service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:25.881751 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106af4bf-648a-4dce-a06b-f2ab3ef586fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:25.882413 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106af4bf-648a-4dce-a06b-f2ab3ef586fb service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:25.882693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106af4bf-648a-4dce-a06b-f2ab3ef586fb service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:25.882965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106af4bf-648a-4dce-a06b-f2ab3ef586fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:25.884768 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-106af4bf-648a-4dce-a06b-f2ab3ef586fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:25.897346 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106af4bf-648a-4dce-a06b-f2ab3ef586fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:25.899115 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106af4bf-648a-4dce-a06b-f2ab3ef586fb service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:25.910184 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1375/2793] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:38:30.309568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15df878-f153-4692-b664-17a33aba8487 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:30.311030 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e15df878-f153-4692-b664-17a33aba8487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:30.383088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15df878-f153-4692-b664-17a33aba8487 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:30.388469 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ec1f7-d37a-4a98-9fd0-b87f9945f51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:30.389388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ec1f7-d37a-4a98-9fd0-b87f9945f51d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:30.389679 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ec1f7-d37a-4a98-9fd0-b87f9945f51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:30.389961 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ec1f7-d37a-4a98-9fd0-b87f9945f51d admin admin] PATH_INFO: `/v3/users/c576d56b32f24e78a4f717df1e0bbd3e` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:30.391508 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ec1f7-d37a-4a98-9fd0-b87f9945f51d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c576d56b32f24e78a4f717df1e0bbd3e)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:30.393549 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ec1f7-d37a-4a98-9fd0-b87f9945f51d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:30.440126 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-e30ec1f7-d37a-4a98-9fd0-b87f9945f51d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c576d56b32f24e78a4f717df1e0bbd3e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:30.452475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-e30ec1f7-d37a-4a98-9fd0-b87f9945f51d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c576d56b32f24e78a4f717df1e0bbd3e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:30.453161 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-e30ec1f7-d37a-4a98-9fd0-b87f9945f51d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c576d56b32f24e78a4f717df1e0bbd3e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:30.460372 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-e30ec1f7-d37a-4a98-9fd0-b87f9945f51d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c576d56b32f24e78a4f717df1e0bbd3e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:30.460983 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-e30ec1f7-d37a-4a98-9fd0-b87f9945f51d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c576d56b32f24e78a4f717df1e0bbd3e'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:30.473608 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1419/2794] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:38:30 2022] DELETE /identity/v3/users/c576d56b32f24e78a4f717df1e0bbd3e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:38:30.478143 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e991a-07fe-49d7-a896-3341064cda28 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:30.478659 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e991a-07fe-49d7-a896-3341064cda28 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:30.478808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e991a-07fe-49d7-a896-3341064cda28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:30.496345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-852e991a-07fe-49d7-a896-3341064cda28 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:30.497491 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-852e991a-07fe-49d7-a896-3341064cda28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:30.557642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1376/2795] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:30.566893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ec1f7-d37a-4a98-9fd0-b87f9945f51d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:30.568373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e30ec1f7-d37a-4a98-9fd0-b87f9945f51d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:30.631543 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ec1f7-d37a-4a98-9fd0-b87f9945f51d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:30.635391 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeaeec6-43f1-4759-996f-a62f52225929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:30.636080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeaeec6-43f1-4759-996f-a62f52225929 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:30.636363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeaeec6-43f1-4759-996f-a62f52225929 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:30.636641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeaeec6-43f1-4759-996f-a62f52225929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:30.648314 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeaeec6-43f1-4759-996f-a62f52225929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:30.649952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeaeec6-43f1-4759-996f-a62f52225929 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:30.660577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1420/2796] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:38:30.937469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e991a-07fe-49d7-a896-3341064cda28 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:30.939439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-852e991a-07fe-49d7-a896-3341064cda28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:30.955552 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e991a-07fe-49d7-a896-3341064cda28 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:30.960037 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0ab68-0157-4391-b63c-5009e5b7e9d0 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:30.960326 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0ab68-0157-4391-b63c-5009e5b7e9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:30.960633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff1407-fa1b-4f9d-a808-0758d5d9950d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:30.960760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0ab68-0157-4391-b63c-5009e5b7e9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:30.961679 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff1407-fa1b-4f9d-a808-0758d5d9950d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:30.962541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff1407-fa1b-4f9d-a808-0758d5d9950d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:30.962816 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff1407-fa1b-4f9d-a808-0758d5d9950d admin admin] PATH_INFO: `/v3/projects/5789bbb2be434abd89ed8c8f3842945a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:30.964295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff1407-fa1b-4f9d-a808-0758d5d9950d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5789bbb2be434abd89ed8c8f3842945a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:30.966853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff1407-fa1b-4f9d-a808-0758d5d9950d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:30.989795 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-6af0ab68-0157-4391-b63c-5009e5b7e9d0 None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:30.991186 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-6af0ab68-0157-4391-b63c-5009e5b7e9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:31.001796 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:31.022067 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-75ff1407-fa1b-4f9d-a808-0758d5d9950d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5789bbb2be434abd89ed8c8f3842945a', 'request_id': 'req-75ff1407-fa1b-4f9d-a808-0758d5d9950d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:31.032217 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-75ff1407-fa1b-4f9d-a808-0758d5d9950d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5789bbb2be434abd89ed8c8f3842945a', 'request_id': 'req-75ff1407-fa1b-4f9d-a808-0758d5d9950d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:31.039187 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-75ff1407-fa1b-4f9d-a808-0758d5d9950d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5789bbb2be434abd89ed8c8f3842945a', 'request_id': 'req-75ff1407-fa1b-4f9d-a808-0758d5d9950d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:31.040444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:31.060011 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-75ff1407-fa1b-4f9d-a808-0758d5d9950d admin admin] The token cache is being invalidate because project 5789bbb2be434abd89ed8c8f3842945a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:38:31.060622 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-75ff1407-fa1b-4f9d-a808-0758d5d9950d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:31.063297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-75ff1407-fa1b-4f9d-a808-0758d5d9950d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:31.065304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1377/2797] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:38:30 2022] DELETE /identity/v3/projects/5789bbb2be434abd89ed8c8f3842945a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:31.070445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d00117-7726-4113-be91-0c8d46bc3dd4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:31.070445 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d00117-7726-4113-be91-0c8d46bc3dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:31.070738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d00117-7726-4113-be91-0c8d46bc3dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:31.094780 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1421/2798] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:38:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:38:31.095056 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f2d00117-7726-4113-be91-0c8d46bc3dd4 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:31.095987 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f2d00117-7726-4113-be91-0c8d46bc3dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:31.156186 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0ab68-0157-4391-b63c-5009e5b7e9d0 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:31.157990 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-6af0ab68-0157-4391-b63c-5009e5b7e9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:31.165178 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1378/2799] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:38:31.173905 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3215da-6bff-4047-92ee-ed4e47199a5f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:31.174173 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3215da-6bff-4047-92ee-ed4e47199a5f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:31.174536 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3215da-6bff-4047-92ee-ed4e47199a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:31.193869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7d3215da-6bff-4047-92ee-ed4e47199a5f None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:31.195114 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7d3215da-6bff-4047-92ee-ed4e47199a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:31.203966 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1379/2800] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:38:31.208028 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3215da-6bff-4047-92ee-ed4e47199a5f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:31.217469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3215da-6bff-4047-92ee-ed4e47199a5f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:31.222589 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de8860-5ccb-404f-8904-206f22cca684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:31.223231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de8860-5ccb-404f-8904-206f22cca684 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:31.223537 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de8860-5ccb-404f-8904-206f22cca684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:31.223813 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de8860-5ccb-404f-8904-206f22cca684 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:31.224541 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de8860-5ccb-404f-8904-206f22cca684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:38:31.224845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de8860-5ccb-404f-8904-206f22cca684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:31.226480 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de8860-5ccb-404f-8904-206f22cca684 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:31.238393 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0ab68-0157-4391-b63c-5009e5b7e9d0 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:31.239703 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1380/2801] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 15:38:31.243829 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de8860-5ccb-404f-8904-206f22cca684 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:31.254297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de8860-5ccb-404f-8904-206f22cca684 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:31.258963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc034c5-62f0-4160-900a-4addeacf7387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:31.259609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc034c5-62f0-4160-900a-4addeacf7387 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:31.259888 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc034c5-62f0-4160-900a-4addeacf7387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:31.260794 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc034c5-62f0-4160-900a-4addeacf7387 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:31.261586 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc034c5-62f0-4160-900a-4addeacf7387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:31.263469 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc034c5-62f0-4160-900a-4addeacf7387 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:31.293662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1381/2802] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:38:31 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:38:31.294674 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc034c5-62f0-4160-900a-4addeacf7387 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:31.311557 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca3577-5de2-4a90-a0f0-dbfb12e603fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:31.312213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca3577-5de2-4a90-a0f0-dbfb12e603fa service swift] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:31.312493 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca3577-5de2-4a90-a0f0-dbfb12e603fa service swift] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:31.312769 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca3577-5de2-4a90-a0f0-dbfb12e603fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:31.358806 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc034c5-62f0-4160-900a-4addeacf7387 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:31.363550 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c864f4-d3b2-45d8-b4e3-1dd0c62f8a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:31.364218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c864f4-d3b2-45d8-b4e3-1dd0c62f8a40 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:31.364500 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c864f4-d3b2-45d8-b4e3-1dd0c62f8a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:31.364780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c864f4-d3b2-45d8-b4e3-1dd0c62f8a40 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:31.365490 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c864f4-d3b2-45d8-b4e3-1dd0c62f8a40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:31.366779 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:31.367408 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c864f4-d3b2-45d8-b4e3-1dd0c62f8a40 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:31.370916 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca3577-5de2-4a90-a0f0-dbfb12e603fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:31.373059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca3577-5de2-4a90-a0f0-dbfb12e603fa service swift] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:31.383324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1422/2803] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:38:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 19 15:38:31.392522 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1382/2804] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:38:31 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:38:31.393578 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca3577-5de2-4a90-a0f0-dbfb12e603fa service swift] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:31.404655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca3577-5de2-4a90-a0f0-dbfb12e603fa service swift] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:31.409568 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7b3b3-66a1-45b6-b150-eec751934ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:31.410239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7b3b3-66a1-45b6-b150-eec751934ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:31.410523 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7b3b3-66a1-45b6-b150-eec751934ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:31.410804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7b3b3-66a1-45b6-b150-eec751934ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:31.411553 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7b3b3-66a1-45b6-b150-eec751934ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:31.413160 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7b3b3-66a1-45b6-b150-eec751934ca0 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:31.424936 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1423/2805] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:38:31.428898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c864f4-d3b2-45d8-b4e3-1dd0c62f8a40 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:31.439938 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c864f4-d3b2-45d8-b4e3-1dd0c62f8a40 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:31.444693 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a7805-2f6d-4572-954a-3128b0ce1981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:31.445397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a7805-2f6d-4572-954a-3128b0ce1981 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:31.445686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a7805-2f6d-4572-954a-3128b0ce1981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:31.445968 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a7805-2f6d-4572-954a-3128b0ce1981 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:31.446703 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a7805-2f6d-4572-954a-3128b0ce1981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:31.449224 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a7805-2f6d-4572-954a-3128b0ce1981 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:31.460311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1383/2806] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:38:31.465233 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7b3b3-66a1-45b6-b150-eec751934ca0 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:31.475793 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7b3b3-66a1-45b6-b150-eec751934ca0 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:31.481937 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130d37c-dc30-4b39-8f80-9502ed0b7925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:31.482727 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130d37c-dc30-4b39-8f80-9502ed0b7925 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:31.483019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130d37c-dc30-4b39-8f80-9502ed0b7925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:31.483291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130d37c-dc30-4b39-8f80-9502ed0b7925 admin admin] PATH_INFO: `/v3/projects/a1e0f23ca7114ecea8330a41af830c22/users/746f7b2b60694c8a87a80f8801204345/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:31.499308 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130d37c-dc30-4b39-8f80-9502ed0b7925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1e0f23ca7114ecea8330a41af830c22, user_id=746f7b2b60694c8a87a80f8801204345, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:31.502008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130d37c-dc30-4b39-8f80-9502ed0b7925 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:31.516044 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1424/2807] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:38:31 2022] PUT /identity/v3/projects/a1e0f23ca7114ecea8330a41af830c22/users/746f7b2b60694c8a87a80f8801204345/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:38:31.521304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdcc5d-1382-408f-8615-3c65c2a60142 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:31.521773 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdcc5d-1382-408f-8615-3c65c2a60142 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:31.521887 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdcc5d-1382-408f-8615-3c65c2a60142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:31.539712 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ecfdcc5d-1382-408f-8615-3c65c2a60142 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:31.540900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ecfdcc5d-1382-408f-8615-3c65c2a60142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:31.567127 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:31.604876 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1384/2808] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:31.614594 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130d37c-dc30-4b39-8f80-9502ed0b7925 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:31.616312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c130d37c-dc30-4b39-8f80-9502ed0b7925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:31.697838 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130d37c-dc30-4b39-8f80-9502ed0b7925 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:31.701608 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db5a9a-addb-4bb0-9e29-dc608aed81e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:31.702324 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db5a9a-addb-4bb0-9e29-dc608aed81e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:31.702605 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db5a9a-addb-4bb0-9e29-dc608aed81e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:31.702877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db5a9a-addb-4bb0-9e29-dc608aed81e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:31.714612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db5a9a-addb-4bb0-9e29-dc608aed81e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:31.716100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db5a9a-addb-4bb0-9e29-dc608aed81e6 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:31.726014 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1425/2809] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:38:31.920503 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8721aad-a4e2-403b-95d0-dc9958bcc1e1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:31.920856 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8721aad-a4e2-403b-95d0-dc9958bcc1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:31.921219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8721aad-a4e2-403b-95d0-dc9958bcc1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:31.939222 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a8721aad-a4e2-403b-95d0-dc9958bcc1e1 None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:31.940430 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a8721aad-a4e2-403b-95d0-dc9958bcc1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:31.996767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1385/2810] 10.222.0.42 () {58 vars in 967 bytes} [Tue Jul 19 15:38:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:32.009213 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db5a9a-addb-4bb0-9e29-dc608aed81e6 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:32.095201 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db5a9a-addb-4bb0-9e29-dc608aed81e6 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:32.102611 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4419c4c-7872-41c1-b5b0-053720fdfb6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:32.103266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4419c4c-7872-41c1-b5b0-053720fdfb6a service swift] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:32.103547 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4419c4c-7872-41c1-b5b0-053720fdfb6a service swift] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:32.103967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4419c4c-7872-41c1-b5b0-053720fdfb6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:32.113188 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4419c4c-7872-41c1-b5b0-053720fdfb6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:32.114778 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4419c4c-7872-41c1-b5b0-053720fdfb6a service swift] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:32.124240 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1426/2811] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:38:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 19 15:38:32.335427 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049af3ed-d8f3-433f-8b1b-9a021ce8cea0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:32.335734 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049af3ed-d8f3-433f-8b1b-9a021ce8cea0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:32.336193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049af3ed-d8f3-433f-8b1b-9a021ce8cea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:32.354922 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-049af3ed-d8f3-433f-8b1b-9a021ce8cea0 None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:32.356317 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-049af3ed-d8f3-433f-8b1b-9a021ce8cea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:32.380163 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1386/2812] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:38:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:32.386456 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc0cbf-976e-441b-a81e-a61b4c173720 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:32.386952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc0cbf-976e-441b-a81e-a61b4c173720 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:32.387040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc0cbf-976e-441b-a81e-a61b4c173720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:32.394191 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049af3ed-d8f3-433f-8b1b-9a021ce8cea0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:32.408543 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049af3ed-d8f3-433f-8b1b-9a021ce8cea0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:32.413667 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9789001-b0bf-4072-ad9d-35d866c18ace service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:32.414319 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9789001-b0bf-4072-ad9d-35d866c18ace service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:32.414604 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9789001-b0bf-4072-ad9d-35d866c18ace service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:32.414884 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9789001-b0bf-4072-ad9d-35d866c18ace service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:32.420208 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-02fc0cbf-976e-441b-a81e-a61b4c173720 None None] MFA Rules not processed for user `746f7b2b60694c8a87a80f8801204345`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:32.421731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-02fc0cbf-976e-441b-a81e-a61b4c173720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:32.424676 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9789001-b0bf-4072-ad9d-35d866c18ace service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:32.426313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9789001-b0bf-4072-ad9d-35d866c18ace service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:32.435792 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1387/2813] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:38:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:38:32.450871 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:32.508641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1427/2814] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:32.520318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9789001-b0bf-4072-ad9d-35d866c18ace service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:32.521850 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b9789001-b0bf-4072-ad9d-35d866c18ace service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:32.592144 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9789001-b0bf-4072-ad9d-35d866c18ace service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:32.597058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a16ca-dbdf-4fcd-8977-f47b7b905b2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:32.597847 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a16ca-dbdf-4fcd-8977-f47b7b905b2d service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:32.598333 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a16ca-dbdf-4fcd-8977-f47b7b905b2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:32.598664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a16ca-dbdf-4fcd-8977-f47b7b905b2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:32.610117 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a16ca-dbdf-4fcd-8977-f47b7b905b2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:32.611722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a16ca-dbdf-4fcd-8977-f47b7b905b2d service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:32.621634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1388/2815] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:38:32.808259 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68356040-1dc7-460c-887c-ab8271fdfcd2 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:32.808689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68356040-1dc7-460c-887c-ab8271fdfcd2 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:32.809068 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68356040-1dc7-460c-887c-ab8271fdfcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:32.829214 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-68356040-1dc7-460c-887c-ab8271fdfcd2 None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:32.830421 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-68356040-1dc7-460c-887c-ab8271fdfcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:32.837803 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1428/2816] 10.222.0.42 () {58 vars in 967 bytes} [Tue Jul 19 15:38:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:32.850766 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a16ca-dbdf-4fcd-8977-f47b7b905b2d service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:32.863442 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a16ca-dbdf-4fcd-8977-f47b7b905b2d service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:32.874062 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c58a2-03d9-429e-bbc2-2e4549ea0cfe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:32.874732 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c58a2-03d9-429e-bbc2-2e4549ea0cfe service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:32.875015 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c58a2-03d9-429e-bbc2-2e4549ea0cfe service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:32.875355 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c58a2-03d9-429e-bbc2-2e4549ea0cfe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:32.886473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c58a2-03d9-429e-bbc2-2e4549ea0cfe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:32.888245 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c58a2-03d9-429e-bbc2-2e4549ea0cfe service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:32.897564 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1389/2817] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:38:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:38:33.994413 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3063f50-10e9-4831-8451-c54e6259e034 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:33.994919 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3063f50-10e9-4831-8451-c54e6259e034 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:33.995080 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3063f50-10e9-4831-8451-c54e6259e034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:34.012677 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-e3063f50-10e9-4831-8451-c54e6259e034 None None] MFA Rules not processed for user `746f7b2b60694c8a87a80f8801204345`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:34.013906 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e3063f50-10e9-4831-8451-c54e6259e034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:34.020952 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1429/2818] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:35.000470 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c58a2-03d9-429e-bbc2-2e4549ea0cfe service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:35.002033 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c72c58a2-03d9-429e-bbc2-2e4549ea0cfe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:35.017097 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c58a2-03d9-429e-bbc2-2e4549ea0cfe service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:35.021984 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cc5c6-8275-417d-b6c6-e8f1c077ae7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:35.022851 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cc5c6-8275-417d-b6c6-e8f1c077ae7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:35.023348 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cc5c6-8275-417d-b6c6-e8f1c077ae7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:35.023647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cc5c6-8275-417d-b6c6-e8f1c077ae7b admin admin] PATH_INFO: `/v3/users/d63df11e4d4344f3b1f6434c4e4a050a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:35.024954 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258cc5c6-8275-417d-b6c6-e8f1c077ae7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d63df11e4d4344f3b1f6434c4e4a050a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:35.027809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258cc5c6-8275-417d-b6c6-e8f1c077ae7b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:35.073269 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:35.079931 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-258cc5c6-8275-417d-b6c6-e8f1c077ae7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd63df11e4d4344f3b1f6434c4e4a050a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:35.094523 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-258cc5c6-8275-417d-b6c6-e8f1c077ae7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd63df11e4d4344f3b1f6434c4e4a050a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:35.095288 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-258cc5c6-8275-417d-b6c6-e8f1c077ae7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd63df11e4d4344f3b1f6434c4e4a050a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:35.095974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-258cc5c6-8275-417d-b6c6-e8f1c077ae7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd63df11e4d4344f3b1f6434c4e4a050a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:35.103816 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-258cc5c6-8275-417d-b6c6-e8f1c077ae7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd63df11e4d4344f3b1f6434c4e4a050a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:35.118719 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1390/2819] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:38:34 2022] DELETE /identity/v3/users/d63df11e4d4344f3b1f6434c4e4a050a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:35.123510 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf070a2-1ecc-4c7c-ae3e-eb085c5e0329 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:35.123977 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf070a2-1ecc-4c7c-ae3e-eb085c5e0329 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:35.124095 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf070a2-1ecc-4c7c-ae3e-eb085c5e0329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:35.144566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-cbf070a2-1ecc-4c7c-ae3e-eb085c5e0329 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:35.145815 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cbf070a2-1ecc-4c7c-ae3e-eb085c5e0329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:35.210889 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1430/2820] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:35.226088 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cc5c6-8275-417d-b6c6-e8f1c077ae7b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:35.227623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-258cc5c6-8275-417d-b6c6-e8f1c077ae7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:35.229076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf070a2-1ecc-4c7c-ae3e-eb085c5e0329 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:35.230649 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cbf070a2-1ecc-4c7c-ae3e-eb085c5e0329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:35.293477 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf070a2-1ecc-4c7c-ae3e-eb085c5e0329 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:35.297426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f6469-076a-4337-985f-d7e8480523e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:35.298086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f6469-076a-4337-985f-d7e8480523e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:35.298373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f6469-076a-4337-985f-d7e8480523e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:35.298642 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f6469-076a-4337-985f-d7e8480523e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:35.300453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-a44f6469-076a-4337-985f-d7e8480523e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:35.314633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cc5c6-8275-417d-b6c6-e8f1c077ae7b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:35.319282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3271e-e8e2-4e2e-9da8-49bda9d146a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:35.319951 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3271e-e8e2-4e2e-9da8-49bda9d146a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:35.320253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3271e-e8e2-4e2e-9da8-49bda9d146a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:35.320514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3271e-e8e2-4e2e-9da8-49bda9d146a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:35.332123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3271e-e8e2-4e2e-9da8-49bda9d146a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:35.333664 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3271e-e8e2-4e2e-9da8-49bda9d146a0 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:35.344353 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1391/2821] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:38:35.370451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f6469-076a-4337-985f-d7e8480523e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:35.372234 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f6469-076a-4337-985f-d7e8480523e3 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:35.384196 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1431/2822] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:38:35.519537 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3271e-e8e2-4e2e-9da8-49bda9d146a0 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:35.521016 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-41f3271e-e8e2-4e2e-9da8-49bda9d146a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:35.550009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3271e-e8e2-4e2e-9da8-49bda9d146a0 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:35.559355 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbe147-6df8-450f-b3de-2f0f0602cf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:35.560331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbe147-6df8-450f-b3de-2f0f0602cf8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:35.560687 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbe147-6df8-450f-b3de-2f0f0602cf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:35.561049 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbe147-6df8-450f-b3de-2f0f0602cf8a admin admin] PATH_INFO: `/v3/projects/2e4fe057576e4bcba6647814498ca197` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:35.562361 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbe147-6df8-450f-b3de-2f0f0602cf8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e4fe057576e4bcba6647814498ca197)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:35.564566 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbe147-6df8-450f-b3de-2f0f0602cf8a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:35.618647 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5fcbe147-6df8-450f-b3de-2f0f0602cf8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e4fe057576e4bcba6647814498ca197', 'request_id': 'req-5fcbe147-6df8-450f-b3de-2f0f0602cf8a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:35.630380 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5fcbe147-6df8-450f-b3de-2f0f0602cf8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e4fe057576e4bcba6647814498ca197', 'request_id': 'req-5fcbe147-6df8-450f-b3de-2f0f0602cf8a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:35.638397 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5fcbe147-6df8-450f-b3de-2f0f0602cf8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e4fe057576e4bcba6647814498ca197', 'request_id': 'req-5fcbe147-6df8-450f-b3de-2f0f0602cf8a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:35.652443 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5fcbe147-6df8-450f-b3de-2f0f0602cf8a admin admin] The token cache is being invalidate because project 2e4fe057576e4bcba6647814498ca197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:38:35.652869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5fcbe147-6df8-450f-b3de-2f0f0602cf8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:35.653662 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5fcbe147-6df8-450f-b3de-2f0f0602cf8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:35.659588 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1392/2823] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:38:35 2022] DELETE /identity/v3/projects/2e4fe057576e4bcba6647814498ca197 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:35.661309 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34537197-2029-48fb-9644-e596828c28f9 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:35.661685 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34537197-2029-48fb-9644-e596828c28f9 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:35.662026 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34537197-2029-48fb-9644-e596828c28f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:35.672232 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:35.687978 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-34537197-2029-48fb-9644-e596828c28f9 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:35.689702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-34537197-2029-48fb-9644-e596828c28f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:35.778921 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1432/2824] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 15:38:35.781762 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449cd0c-5681-461c-8be6-7ac47c210648 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:35.781762 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449cd0c-5681-461c-8be6-7ac47c210648 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:35.781762 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449cd0c-5681-461c-8be6-7ac47c210648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:35.804137 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c449cd0c-5681-461c-8be6-7ac47c210648 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:35.806158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c449cd0c-5681-461c-8be6-7ac47c210648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:35.826360 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1393/2825] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:35.827574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34537197-2029-48fb-9644-e596828c28f9 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:35.842056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34537197-2029-48fb-9644-e596828c28f9 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:35.848013 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225ef70-2ef4-4095-8f32-b8dfa94d9117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:35.848706 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225ef70-2ef4-4095-8f32-b8dfa94d9117 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:35.848931 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225ef70-2ef4-4095-8f32-b8dfa94d9117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:35.849514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225ef70-2ef4-4095-8f32-b8dfa94d9117 admin admin] PATH_INFO: `/v3/domains` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:35.850269 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225ef70-2ef4-4095-8f32-b8dfa94d9117 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80271) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:38:35.852877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225ef70-2ef4-4095-8f32-b8dfa94d9117 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:35.854639 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225ef70-2ef4-4095-8f32-b8dfa94d9117 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:35.873290 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1433/2826] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:38:35.874060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449cd0c-5681-461c-8be6-7ac47c210648 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:35.887318 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449cd0c-5681-461c-8be6-7ac47c210648 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:35.895345 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a0db0-0a2c-4c4b-bb2e-8c128e1db3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:35.896110 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a0db0-0a2c-4c4b-bb2e-8c128e1db3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:35.896451 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a0db0-0a2c-4c4b-bb2e-8c128e1db3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:35.896796 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a0db0-0a2c-4c4b-bb2e-8c128e1db3e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:35.897788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a0db0-0a2c-4c4b-bb2e-8c128e1db3e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:35.900271 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a0db0-0a2c-4c4b-bb2e-8c128e1db3e1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:35.933084 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1394/2827] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:38:35 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:38:35.937426 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225ef70-2ef4-4095-8f32-b8dfa94d9117 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:35.948178 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225ef70-2ef4-4095-8f32-b8dfa94d9117 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:36.033272 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a251db2-608e-48cf-bb3e-9c4577ad320d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:36.034087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a251db2-608e-48cf-bb3e-9c4577ad320d admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:36.034379 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a251db2-608e-48cf-bb3e-9c4577ad320d admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:36.034708 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a251db2-608e-48cf-bb3e-9c4577ad320d admin admin] PATH_INFO: `/v3/users` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:36.035567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a251db2-608e-48cf-bb3e-9c4577ad320d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:36.037925 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a251db2-608e-48cf-bb3e-9c4577ad320d admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:36.072086 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1434/2828] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:38:35 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:38:36.077535 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a0db0-0a2c-4c4b-bb2e-8c128e1db3e1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:36.095284 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a0db0-0a2c-4c4b-bb2e-8c128e1db3e1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:36.100975 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444f454-0546-4c33-bf6e-0d6478f7b149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:36.101740 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444f454-0546-4c33-bf6e-0d6478f7b149 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:36.102108 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444f454-0546-4c33-bf6e-0d6478f7b149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:36.102458 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444f454-0546-4c33-bf6e-0d6478f7b149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:36.103310 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444f454-0546-4c33-bf6e-0d6478f7b149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:36.105526 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444f454-0546-4c33-bf6e-0d6478f7b149 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:36.122047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1395/2829] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:38:36.125992 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a251db2-608e-48cf-bb3e-9c4577ad320d admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:36.137464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a251db2-608e-48cf-bb3e-9c4577ad320d admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:36.143292 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5848a1a-1918-45ed-b269-dc4995efedce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:36.143968 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5848a1a-1918-45ed-b269-dc4995efedce admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:36.144259 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5848a1a-1918-45ed-b269-dc4995efedce admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:36.144534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5848a1a-1918-45ed-b269-dc4995efedce admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:36.145521 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5848a1a-1918-45ed-b269-dc4995efedce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:36.146916 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5848a1a-1918-45ed-b269-dc4995efedce admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:36.158530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1435/2830] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 15:38:36.162939 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444f454-0546-4c33-bf6e-0d6478f7b149 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:36.170087 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:36.173945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444f454-0546-4c33-bf6e-0d6478f7b149 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:36.182465 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79888742-6407-4358-833b-73de5796c6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:36.183466 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79888742-6407-4358-833b-73de5796c6d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:36.184141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79888742-6407-4358-833b-73de5796c6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:36.184141 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79888742-6407-4358-833b-73de5796c6d4 admin admin] PATH_INFO: `/v3/projects/05857a00bdb14c54800b5e0f55ed27fc/users/15af8262a49946739a79f12ec7f8b15d/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:36.204796 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79888742-6407-4358-833b-73de5796c6d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05857a00bdb14c54800b5e0f55ed27fc, user_id=15af8262a49946739a79f12ec7f8b15d, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:36.208399 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79888742-6407-4358-833b-73de5796c6d4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:36.224283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1396/2831] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:38:36 2022] PUT /identity/v3/projects/05857a00bdb14c54800b5e0f55ed27fc/users/15af8262a49946739a79f12ec7f8b15d/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:36.229211 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959b4a5-e52b-40c9-91db-963dc6759873 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:36.229512 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959b4a5-e52b-40c9-91db-963dc6759873 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:36.229791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959b4a5-e52b-40c9-91db-963dc6759873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:36.250943 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-b959b4a5-e52b-40c9-91db-963dc6759873 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:36.252478 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b959b4a5-e52b-40c9-91db-963dc6759873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:36.330401 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79888742-6407-4358-833b-73de5796c6d4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:36.332214 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-79888742-6407-4358-833b-73de5796c6d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:36.344560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1436/2832] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:36.356061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959b4a5-e52b-40c9-91db-963dc6759873 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:36.357919 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-b959b4a5-e52b-40c9-91db-963dc6759873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:36.380327 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:36.455412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959b4a5-e52b-40c9-91db-963dc6759873 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:36.459396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b73c5b-e1ae-4511-90f9-e299173cd5b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:36.460075 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b73c5b-e1ae-4511-90f9-e299173cd5b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:36.460364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b73c5b-e1ae-4511-90f9-e299173cd5b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:36.460877 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b73c5b-e1ae-4511-90f9-e299173cd5b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:36.473546 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b73c5b-e1ae-4511-90f9-e299173cd5b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:36.475052 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b73c5b-e1ae-4511-90f9-e299173cd5b6 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:36.487133 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1437/2833] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:38:36.564656 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79888742-6407-4358-833b-73de5796c6d4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:36.581591 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802774f7-869b-4255-8e61-0c4472abd38f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:36.581819 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802774f7-869b-4255-8e61-0c4472abd38f service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:36.582096 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802774f7-869b-4255-8e61-0c4472abd38f service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:36.582372 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802774f7-869b-4255-8e61-0c4472abd38f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:36.584234 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-802774f7-869b-4255-8e61-0c4472abd38f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:36.603974 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802774f7-869b-4255-8e61-0c4472abd38f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:36.605451 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802774f7-869b-4255-8e61-0c4472abd38f service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:36.618756 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1397/2834] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:38:42.056494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b73c5b-e1ae-4511-90f9-e299173cd5b6 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:42.058127 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-62b73c5b-e1ae-4511-90f9-e299173cd5b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:42.071264 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b73c5b-e1ae-4511-90f9-e299173cd5b6 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:42.075807 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8db5e-b2a8-4c55-8d87-024b15376207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:42.076574 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8db5e-b2a8-4c55-8d87-024b15376207 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:42.076958 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8db5e-b2a8-4c55-8d87-024b15376207 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:42.077362 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8db5e-b2a8-4c55-8d87-024b15376207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:42.079310 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-55d8db5e-b2a8-4c55-8d87-024b15376207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:42.094828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8db5e-b2a8-4c55-8d87-024b15376207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:42.096401 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8db5e-b2a8-4c55-8d87-024b15376207 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:42.107438 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1438/2835] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:38:42.456382 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935490f-eca6-46f3-9e50-1a8c0ab4587f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:42.457392 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935490f-eca6-46f3-9e50-1a8c0ab4587f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:42.457918 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935490f-eca6-46f3-9e50-1a8c0ab4587f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:42.497454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e935490f-eca6-46f3-9e50-1a8c0ab4587f None None] MFA Rules not processed for user `15af8262a49946739a79f12ec7f8b15d`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:42.500890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e935490f-eca6-46f3-9e50-1a8c0ab4587f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:42.612551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1398/2836] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:42.617122 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8db5e-b2a8-4c55-8d87-024b15376207 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:42.628363 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8db5e-b2a8-4c55-8d87-024b15376207 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:42.634062 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdeef8d-78b5-4110-8181-2bb5717d9fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:42.635371 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdeef8d-78b5-4110-8181-2bb5717d9fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:42.636013 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdeef8d-78b5-4110-8181-2bb5717d9fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:42.636013 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdeef8d-78b5-4110-8181-2bb5717d9fa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:42.636520 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdeef8d-78b5-4110-8181-2bb5717d9fa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:42.638534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdeef8d-78b5-4110-8181-2bb5717d9fa7 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:42.672100 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1439/2837] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:38:42 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:38:42.673633 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935490f-eca6-46f3-9e50-1a8c0ab4587f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:42.686626 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935490f-eca6-46f3-9e50-1a8c0ab4587f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:42.731282 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:42.758350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af597c-9484-4c8f-8a12-01ee31c2e9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:42.759198 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af597c-9484-4c8f-8a12-01ee31c2e9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:42.760331 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af597c-9484-4c8f-8a12-01ee31c2e9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:42.760809 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af597c-9484-4c8f-8a12-01ee31c2e9b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:42.761739 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af597c-9484-4c8f-8a12-01ee31c2e9b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:42.763830 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af597c-9484-4c8f-8a12-01ee31c2e9b8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:42.787588 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1399/2838] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:38:42 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:38:42.792693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdeef8d-78b5-4110-8181-2bb5717d9fa7 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:42.803967 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdeef8d-78b5-4110-8181-2bb5717d9fa7 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:42.812466 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e03153-6fd1-4560-8036-0346f531a150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:42.813207 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e03153-6fd1-4560-8036-0346f531a150 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:42.813499 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e03153-6fd1-4560-8036-0346f531a150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:42.813781 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e03153-6fd1-4560-8036-0346f531a150 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:42.814540 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e03153-6fd1-4560-8036-0346f531a150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:42.816191 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e03153-6fd1-4560-8036-0346f531a150 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:42.829949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1440/2839] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:38:42.834324 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af597c-9484-4c8f-8a12-01ee31c2e9b8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:42.847800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af597c-9484-4c8f-8a12-01ee31c2e9b8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:42.853066 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0006426-dee3-4798-9112-27e887c43517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:42.854768 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0006426-dee3-4798-9112-27e887c43517 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:42.855123 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0006426-dee3-4798-9112-27e887c43517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:42.855560 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0006426-dee3-4798-9112-27e887c43517 admin admin] PATH_INFO: `/v3/projects/2fa7bda25eec409f88dc10e649241132/users/155dd98501894a1d856de203dfc6689a/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:42.873316 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0006426-dee3-4798-9112-27e887c43517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa7bda25eec409f88dc10e649241132, user_id=155dd98501894a1d856de203dfc6689a, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:42.876092 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0006426-dee3-4798-9112-27e887c43517 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:42.890717 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1400/2840] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:38:42 2022] PUT /identity/v3/projects/2fa7bda25eec409f88dc10e649241132/users/155dd98501894a1d856de203dfc6689a/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:42.894976 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e03153-6fd1-4560-8036-0346f531a150 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:42.906635 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e03153-6fd1-4560-8036-0346f531a150 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:42.965651 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a97dde-09e6-4f96-a342-2c9455570933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:42.966328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a97dde-09e6-4f96-a342-2c9455570933 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:42.966613 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a97dde-09e6-4f96-a342-2c9455570933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:42.966890 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a97dde-09e6-4f96-a342-2c9455570933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:42.967627 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a97dde-09e6-4f96-a342-2c9455570933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:42.969263 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a97dde-09e6-4f96-a342-2c9455570933 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:42.983167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1441/2841] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:38:42.984511 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0006426-dee3-4798-9112-27e887c43517 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:42.997290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0006426-dee3-4798-9112-27e887c43517 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:43.002823 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674f463-95da-4b34-bfd8-ec7a6205fc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:43.003739 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674f463-95da-4b34-bfd8-ec7a6205fc54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:43.004014 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674f463-95da-4b34-bfd8-ec7a6205fc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:43.004293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674f463-95da-4b34-bfd8-ec7a6205fc54 admin admin] PATH_INFO: `/v3/projects/2fa7bda25eec409f88dc10e649241132/users/155dd98501894a1d856de203dfc6689a/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:43.006344 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674f463-95da-4b34-bfd8-ec7a6205fc54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa7bda25eec409f88dc10e649241132, user_id=155dd98501894a1d856de203dfc6689a, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:43.008948 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674f463-95da-4b34-bfd8-ec7a6205fc54 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:43.021252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1401/2842] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:38:42 2022] PUT /identity/v3/projects/2fa7bda25eec409f88dc10e649241132/users/155dd98501894a1d856de203dfc6689a/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:49.999167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf520b-3162-450c-8982-18e803be9684 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:49.999167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf520b-3162-450c-8982-18e803be9684 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:49.999167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf520b-3162-450c-8982-18e803be9684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:50.032408 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-30cf520b-3162-450c-8982-18e803be9684 None None] MFA Rules not processed for user `155dd98501894a1d856de203dfc6689a`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:50.034061 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-30cf520b-3162-450c-8982-18e803be9684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:50.104920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:50.135376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1442/2843] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:38:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:50.148730 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674f463-95da-4b34-bfd8-ec7a6205fc54 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:50.152134 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f674f463-95da-4b34-bfd8-ec7a6205fc54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:50.286010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674f463-95da-4b34-bfd8-ec7a6205fc54 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:50.299230 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b21ef-2294-4e3d-a6bb-ecc7bb00daf2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '215c6f6f1aad486f85ce20590d889cbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:50.302891 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b21ef-2294-4e3d-a6bb-ecc7bb00daf2 service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:50.303185 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b21ef-2294-4e3d-a6bb-ecc7bb00daf2 service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:50.303559 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b21ef-2294-4e3d-a6bb-ecc7bb00daf2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:50.480294 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b21ef-2294-4e3d-a6bb-ecc7bb00daf2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:50.482031 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b21ef-2294-4e3d-a6bb-ecc7bb00daf2 service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:50.523280 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1402/2844] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:38:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:38:50.669266 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b542c74-2753-4ee5-9fbc-e5b73f1ee5c6 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:50.669976 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b542c74-2753-4ee5-9fbc-e5b73f1ee5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:50.670455 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b542c74-2753-4ee5-9fbc-e5b73f1ee5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:50.698078 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-8b542c74-2753-4ee5-9fbc-e5b73f1ee5c6 None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:38:50.699321 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-8b542c74-2753-4ee5-9fbc-e5b73f1ee5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:50.792123 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1443/2845] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:38:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:38:50.803994 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b21ef-2294-4e3d-a6bb-ecc7bb00daf2 service glance] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:50.805538 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-876b21ef-2294-4e3d-a6bb-ecc7bb00daf2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:50.883366 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:50.941174 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b21ef-2294-4e3d-a6bb-ecc7bb00daf2 service glance] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:50.947764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55391b1c-9cbd-400e-84e6-727e34339030 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:50.949182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55391b1c-9cbd-400e-84e6-727e34339030 service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:50.949182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55391b1c-9cbd-400e-84e6-727e34339030 service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:50.949514 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55391b1c-9cbd-400e-84e6-727e34339030 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:50.959478 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55391b1c-9cbd-400e-84e6-727e34339030 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:50.961377 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55391b1c-9cbd-400e-84e6-727e34339030 service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:50.973046 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1403/2846] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:38:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:38:53.038368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b542c74-2753-4ee5-9fbc-e5b73f1ee5c6 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:53.041514 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-8b542c74-2753-4ee5-9fbc-e5b73f1ee5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:53.147585 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b542c74-2753-4ee5-9fbc-e5b73f1ee5c6 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:53.162235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569fe87-44b7-4bde-87a3-7ce9560ef0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:53.163095 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569fe87-44b7-4bde-87a3-7ce9560ef0cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:53.163381 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569fe87-44b7-4bde-87a3-7ce9560ef0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:53.163660 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569fe87-44b7-4bde-87a3-7ce9560ef0cb admin admin] PATH_INFO: `/v3/users/dee7cef53eb641bb93c92e3c524c59db` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:53.165657 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569fe87-44b7-4bde-87a3-7ce9560ef0cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=dee7cef53eb641bb93c92e3c524c59db)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:53.167802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569fe87-44b7-4bde-87a3-7ce9560ef0cb admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:53.256222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c569fe87-44b7-4bde-87a3-7ce9560ef0cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dee7cef53eb641bb93c92e3c524c59db'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:53.273189 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c569fe87-44b7-4bde-87a3-7ce9560ef0cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dee7cef53eb641bb93c92e3c524c59db'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:53.277196 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c569fe87-44b7-4bde-87a3-7ce9560ef0cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dee7cef53eb641bb93c92e3c524c59db'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:53.284959 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c569fe87-44b7-4bde-87a3-7ce9560ef0cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dee7cef53eb641bb93c92e3c524c59db'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:53.285671 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c569fe87-44b7-4bde-87a3-7ce9560ef0cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dee7cef53eb641bb93c92e3c524c59db'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:53.301934 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1444/2847] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:38:53 2022] DELETE /identity/v3/users/dee7cef53eb641bb93c92e3c524c59db => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:53.608866 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55391b1c-9cbd-400e-84e6-727e34339030 service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:53.625802 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55391b1c-9cbd-400e-84e6-727e34339030 service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:53.722592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aabf3f-226f-4124-8012-bf1ec2dfc753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:53.723673 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aabf3f-226f-4124-8012-bf1ec2dfc753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:53.729401 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aabf3f-226f-4124-8012-bf1ec2dfc753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:53.729727 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aabf3f-226f-4124-8012-bf1ec2dfc753 admin admin] PATH_INFO: `/v3/projects/286e8f4ce0854bf3a8563bcea08ae93a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:53.731956 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aabf3f-226f-4124-8012-bf1ec2dfc753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=286e8f4ce0854bf3a8563bcea08ae93a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:53.733946 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aabf3f-226f-4124-8012-bf1ec2dfc753 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:53.814782 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d9aabf3f-226f-4124-8012-bf1ec2dfc753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '286e8f4ce0854bf3a8563bcea08ae93a', 'request_id': 'req-d9aabf3f-226f-4124-8012-bf1ec2dfc753'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:53.827605 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d9aabf3f-226f-4124-8012-bf1ec2dfc753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '286e8f4ce0854bf3a8563bcea08ae93a', 'request_id': 'req-d9aabf3f-226f-4124-8012-bf1ec2dfc753'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:53.835667 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d9aabf3f-226f-4124-8012-bf1ec2dfc753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '286e8f4ce0854bf3a8563bcea08ae93a', 'request_id': 'req-d9aabf3f-226f-4124-8012-bf1ec2dfc753'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:53.854561 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d9aabf3f-226f-4124-8012-bf1ec2dfc753 admin admin] The token cache is being invalidate because project 286e8f4ce0854bf3a8563bcea08ae93a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:38:53.854869 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d9aabf3f-226f-4124-8012-bf1ec2dfc753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:53.855525 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d9aabf3f-226f-4124-8012-bf1ec2dfc753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:53.860182 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1404/2848] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:38:53 2022] DELETE /identity/v3/projects/286e8f4ce0854bf3a8563bcea08ae93a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:53.861757 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569fe87-44b7-4bde-87a3-7ce9560ef0cb admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:53.863775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-c569fe87-44b7-4bde-87a3-7ce9560ef0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:53.879362 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569fe87-44b7-4bde-87a3-7ce9560ef0cb admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:53.901968 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:53.984620 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55abdb3d-16e4-488e-927d-2bb1923a9d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:53.985655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abdb3d-16e4-488e-927d-2bb1923a9d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:53.986180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abdb3d-16e4-488e-927d-2bb1923a9d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:53.986428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abdb3d-16e4-488e-927d-2bb1923a9d6a admin admin] PATH_INFO: `/v3/users/1577a7a091eb466f9c083d084cc64b5c` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:53.993195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55abdb3d-16e4-488e-927d-2bb1923a9d6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1577a7a091eb466f9c083d084cc64b5c)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:53.995960 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55abdb3d-16e4-488e-927d-2bb1923a9d6a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:54.067216 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-55abdb3d-16e4-488e-927d-2bb1923a9d6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1577a7a091eb466f9c083d084cc64b5c'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:54.091919 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-55abdb3d-16e4-488e-927d-2bb1923a9d6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1577a7a091eb466f9c083d084cc64b5c'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:54.093475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-55abdb3d-16e4-488e-927d-2bb1923a9d6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1577a7a091eb466f9c083d084cc64b5c'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:54.105913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-55abdb3d-16e4-488e-927d-2bb1923a9d6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1577a7a091eb466f9c083d084cc64b5c'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:54.108714 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-55abdb3d-16e4-488e-927d-2bb1923a9d6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1577a7a091eb466f9c083d084cc64b5c'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:54.159619 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1445/2849] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:38:53 2022] DELETE /identity/v3/users/1577a7a091eb466f9c083d084cc64b5c => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:38:54.474746 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aabf3f-226f-4124-8012-bf1ec2dfc753 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:38:54.476724 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d9aabf3f-226f-4124-8012-bf1ec2dfc753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:38:54.508056 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aabf3f-226f-4124-8012-bf1ec2dfc753 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:38:54.646999 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0d0e3-e9e8-46d5-9ff3-e52d949e00d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:38:54.650064 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0d0e3-e9e8-46d5-9ff3-e52d949e00d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:38:54.650064 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0d0e3-e9e8-46d5-9ff3-e52d949e00d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:38:54.650064 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0d0e3-e9e8-46d5-9ff3-e52d949e00d7 admin admin] PATH_INFO: `/v3/projects/fe1038820b5b4754a05ac3bac4e9594e` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:38:54.651151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0d0e3-e9e8-46d5-9ff3-e52d949e00d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe1038820b5b4754a05ac3bac4e9594e)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:38:54.653587 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0d0e3-e9e8-46d5-9ff3-e52d949e00d7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:38:54.657032 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:38:54.781444 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-eee0d0e3-e9e8-46d5-9ff3-e52d949e00d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe1038820b5b4754a05ac3bac4e9594e', 'request_id': 'req-eee0d0e3-e9e8-46d5-9ff3-e52d949e00d7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:54.798652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-eee0d0e3-e9e8-46d5-9ff3-e52d949e00d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe1038820b5b4754a05ac3bac4e9594e', 'request_id': 'req-eee0d0e3-e9e8-46d5-9ff3-e52d949e00d7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:54.828332 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-eee0d0e3-e9e8-46d5-9ff3-e52d949e00d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe1038820b5b4754a05ac3bac4e9594e', 'request_id': 'req-eee0d0e3-e9e8-46d5-9ff3-e52d949e00d7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:54.846642 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-eee0d0e3-e9e8-46d5-9ff3-e52d949e00d7 admin admin] The token cache is being invalidate because project fe1038820b5b4754a05ac3bac4e9594e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:38:54.846933 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-eee0d0e3-e9e8-46d5-9ff3-e52d949e00d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:54.847590 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-eee0d0e3-e9e8-46d5-9ff3-e52d949e00d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:38:54.869289 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1405/2850] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:38:54 2022] DELETE /identity/v3/projects/fe1038820b5b4754a05ac3bac4e9594e => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:39:01.018851 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342ee37-0aa7-46a6-a985-03972bdac29f None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:39:01.019415 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342ee37-0aa7-46a6-a985-03972bdac29f None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:39:01.019415 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342ee37-0aa7-46a6-a985-03972bdac29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:39:01.037642 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-9342ee37-0aa7-46a6-a985-03972bdac29f None None] MFA Rules not processed for user `15af8262a49946739a79f12ec7f8b15d`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:39:01.038861 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9342ee37-0aa7-46a6-a985-03972bdac29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:39:01.097422 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1446/2851] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:39:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:39:02.903992 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448d2be-eac5-4318-9aeb-bd79b8c27633 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:39:02.904538 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448d2be-eac5-4318-9aeb-bd79b8c27633 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:39:02.904651 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448d2be-eac5-4318-9aeb-bd79b8c27633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:39:02.923653 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9448d2be-eac5-4318-9aeb-bd79b8c27633 None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:39:02.924939 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9448d2be-eac5-4318-9aeb-bd79b8c27633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:39:02.982269 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1406/2852] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Jul 19 15:39:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:39:02.994529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342ee37-0aa7-46a6-a985-03972bdac29f None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:39:02.995977 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9342ee37-0aa7-46a6-a985-03972bdac29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:39:03.041720 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:39:03.071855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342ee37-0aa7-46a6-a985-03972bdac29f None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:39:03.077155 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2893d8-8eec-4108-9830-f50b84353e82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:39:03.077902 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2893d8-8eec-4108-9830-f50b84353e82 service swift] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:39:03.078190 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2893d8-8eec-4108-9830-f50b84353e82 service swift] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:39:03.078495 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2893d8-8eec-4108-9830-f50b84353e82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:39:03.089205 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2893d8-8eec-4108-9830-f50b84353e82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:39:03.091647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2893d8-8eec-4108-9830-f50b84353e82 service swift] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:39:03.100847 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1447/2853] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:39:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:39:12.974483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:39:13.097257 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=15 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:40:14.946005 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448d2be-eac5-4318-9aeb-bd79b8c27633 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:40:14.950255 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9448d2be-eac5-4318-9aeb-bd79b8c27633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:40:15.007072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448d2be-eac5-4318-9aeb-bd79b8c27633 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:40:15.012143 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115553e8-10d2-4fe0-9ce0-b320b51a131c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:40:15.012793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115553e8-10d2-4fe0-9ce0-b320b51a131c service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:40:15.013077 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115553e8-10d2-4fe0-9ce0-b320b51a131c service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:40:15.013383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115553e8-10d2-4fe0-9ce0-b320b51a131c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:40:15.015446 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-115553e8-10d2-4fe0-9ce0-b320b51a131c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:40:15.085767 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115553e8-10d2-4fe0-9ce0-b320b51a131c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:40:15.088358 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115553e8-10d2-4fe0-9ce0-b320b51a131c service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:40:15.100006 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1407/2854] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:40:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:40:15.224072 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2893d8-8eec-4108-9830-f50b84353e82 service swift] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:40:15.234076 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2893d8-8eec-4108-9830-f50b84353e82 service swift] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:40:15.237630 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98b699-b286-4537-a238-321a864c1f6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:40:15.238271 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98b699-b286-4537-a238-321a864c1f6c service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:40:15.238551 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98b699-b286-4537-a238-321a864c1f6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:40:15.238825 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98b699-b286-4537-a238-321a864c1f6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:40:15.240598 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ed98b699-b286-4537-a238-321a864c1f6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:40:15.303947 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98b699-b286-4537-a238-321a864c1f6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:40:15.305803 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98b699-b286-4537-a238-321a864c1f6c service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:40:15.317389 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1448/2855] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:40:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:40:18.690542 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115553e8-10d2-4fe0-9ce0-b320b51a131c service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:40:18.700338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115553e8-10d2-4fe0-9ce0-b320b51a131c service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:40:18.704924 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6202b-17ee-4269-bb9f-deda2180bb26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:40:18.705731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6202b-17ee-4269-bb9f-deda2180bb26 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:40:18.706110 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6202b-17ee-4269-bb9f-deda2180bb26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:40:18.706483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6202b-17ee-4269-bb9f-deda2180bb26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:40:18.708483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-57a6202b-17ee-4269-bb9f-deda2180bb26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:40:18.724210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6202b-17ee-4269-bb9f-deda2180bb26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:40:18.725993 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6202b-17ee-4269-bb9f-deda2180bb26 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:40:18.739763 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1408/2856] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:40:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:40:25.311204 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=30 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:40:28.734283 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=60 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:41:19.614509 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98b699-b286-4537-a238-321a864c1f6c service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:41:19.616142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-ed98b699-b286-4537-a238-321a864c1f6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:41:19.652275 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98b699-b286-4537-a238-321a864c1f6c service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:41:19.661937 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed93398-0f64-4588-acd7-20a001512744 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:41:19.662623 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed93398-0f64-4588-acd7-20a001512744 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:41:19.662905 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed93398-0f64-4588-acd7-20a001512744 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:41:19.663184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed93398-0f64-4588-acd7-20a001512744 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:41:19.666631 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5ed93398-0f64-4588-acd7-20a001512744 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:41:19.682912 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed93398-0f64-4588-acd7-20a001512744 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:41:19.684788 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed93398-0f64-4588-acd7-20a001512744 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:41:19.705089 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1449/2857] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:41:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:41:19.769060 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6202b-17ee-4269-bb9f-deda2180bb26 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:41:19.781789 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6202b-17ee-4269-bb9f-deda2180bb26 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:41:19.786832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97dece9-6c0b-4045-970f-8128831b660e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:41:19.788480 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97dece9-6c0b-4045-970f-8128831b660e service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:41:19.789089 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97dece9-6c0b-4045-970f-8128831b660e service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:41:19.790047 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97dece9-6c0b-4045-970f-8128831b660e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:41:19.793825 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e97dece9-6c0b-4045-970f-8128831b660e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:41:19.809671 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97dece9-6c0b-4045-970f-8128831b660e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:41:19.811215 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97dece9-6c0b-4045-970f-8128831b660e service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:41:19.821650 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1409/2858] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:41:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:41:29.692056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=6 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:41:29.816257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:43:27.258693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed93398-0f64-4588-acd7-20a001512744 service nova] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:43:27.260082 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5ed93398-0f64-4588-acd7-20a001512744 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:43:27.330409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed93398-0f64-4588-acd7-20a001512744 service nova] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:43:27.335394 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758161a-49a4-49b8-a627-5fdc3f905cde service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:43:27.336040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758161a-49a4-49b8-a627-5fdc3f905cde service placement] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:43:27.336322 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758161a-49a4-49b8-a627-5fdc3f905cde service placement] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:43:27.336602 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758161a-49a4-49b8-a627-5fdc3f905cde service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:43:27.338388 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1758161a-49a4-49b8-a627-5fdc3f905cde service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:43:27.350181 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758161a-49a4-49b8-a627-5fdc3f905cde service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:43:27.351760 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758161a-49a4-49b8-a627-5fdc3f905cde service placement] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:43:27.362151 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1450/2859] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:43:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:43:32.131414 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97dece9-6c0b-4045-970f-8128831b660e service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:43:32.132800 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e97dece9-6c0b-4045-970f-8128831b660e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:43:32.209350 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97dece9-6c0b-4045-970f-8128831b660e service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:43:32.212965 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871311d-3285-4688-a115-85c65468056b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:43:32.213640 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871311d-3285-4688-a115-85c65468056b service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:43:32.213928 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871311d-3285-4688-a115-85c65468056b service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:43:32.214201 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871311d-3285-4688-a115-85c65468056b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:43:32.215963 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b871311d-3285-4688-a115-85c65468056b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:43:32.275607 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b871311d-3285-4688-a115-85c65468056b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:43:32.277043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b871311d-3285-4688-a115-85c65468056b service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:43:32.286344 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1410/2860] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:43:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:43:37.357457 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=32 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:43:42.282196 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=57 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:43:49.692328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758161a-49a4-49b8-a627-5fdc3f905cde service placement] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:43:49.704655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758161a-49a4-49b8-a627-5fdc3f905cde service placement] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:43:49.711165 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2fe5f-d47f-49a3-9cf1-b672311c02cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:43:49.711804 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2fe5f-d47f-49a3-9cf1-b672311c02cd service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:43:49.712087 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2fe5f-d47f-49a3-9cf1-b672311c02cd service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:43:49.712364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2fe5f-d47f-49a3-9cf1-b672311c02cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:43:49.714304 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-d4b2fe5f-d47f-49a3-9cf1-b672311c02cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:43:49.725093 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2fe5f-d47f-49a3-9cf1-b672311c02cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:43:49.726698 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2fe5f-d47f-49a3-9cf1-b672311c02cd service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:43:49.735530 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1451/2861] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:43:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:43:59.731494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=6 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:45:26.369670 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871311d-3285-4688-a115-85c65468056b service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:45:26.371052 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b871311d-3285-4688-a115-85c65468056b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:45:26.383010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871311d-3285-4688-a115-85c65468056b service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:45:26.386575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660488e6-6d96-4ae6-982c-f3da0beeb68b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:45:26.387218 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660488e6-6d96-4ae6-982c-f3da0beeb68b service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:45:26.387498 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660488e6-6d96-4ae6-982c-f3da0beeb68b service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:45:26.387909 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660488e6-6d96-4ae6-982c-f3da0beeb68b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:45:26.398761 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660488e6-6d96-4ae6-982c-f3da0beeb68b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:45:26.400224 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660488e6-6d96-4ae6-982c-f3da0beeb68b service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:45:26.411785 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1411/2862] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:45:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:45:36.406622 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:46:35.001551 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2fe5f-d47f-49a3-9cf1-b672311c02cd service nova] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:46:35.011008 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2fe5f-d47f-49a3-9cf1-b672311c02cd service nova] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:46:35.015229 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c3272-2da3-4879-9a1a-79a6b19f098d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:46:35.015857 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c3272-2da3-4879-9a1a-79a6b19f098d service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:46:35.016137 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c3272-2da3-4879-9a1a-79a6b19f098d service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:46:35.016412 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c3272-2da3-4879-9a1a-79a6b19f098d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:46:35.023976 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c3272-2da3-4879-9a1a-79a6b19f098d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:46:35.025451 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c3272-2da3-4879-9a1a-79a6b19f098d service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:46:35.033584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1452/2863] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:46:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:46:45.030034 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=6 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:47:22.038018 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660488e6-6d96-4ae6-982c-f3da0beeb68b service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:47:22.046876 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660488e6-6d96-4ae6-982c-f3da0beeb68b service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:47:22.050735 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee39ff3-75ef-4b76-9375-6340072999e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:47:22.051389 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee39ff3-75ef-4b76-9375-6340072999e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:47:22.051669 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee39ff3-75ef-4b76-9375-6340072999e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:47:22.051942 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee39ff3-75ef-4b76-9375-6340072999e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:47:22.062528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee39ff3-75ef-4b76-9375-6340072999e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:47:22.064655 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee39ff3-75ef-4b76-9375-6340072999e8 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:47:22.075779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1412/2864] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:47:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:47:32.070822 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:48:32.106142 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c3272-2da3-4879-9a1a-79a6b19f098d service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:48:32.126840 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c3272-2da3-4879-9a1a-79a6b19f098d service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:48:32.130491 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6a3fe-e7e5-4503-9cce-1f7277483711 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:48:32.131131 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6a3fe-e7e5-4503-9cce-1f7277483711 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:48:32.131417 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6a3fe-e7e5-4503-9cce-1f7277483711 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:48:32.131693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6a3fe-e7e5-4503-9cce-1f7277483711 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:48:32.156710 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6a3fe-e7e5-4503-9cce-1f7277483711 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:48:32.158179 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6a3fe-e7e5-4503-9cce-1f7277483711 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:48:32.167526 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1453/2865] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:48:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:48:32.267677 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee39ff3-75ef-4b76-9375-6340072999e8 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:48:32.269381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9ee39ff3-75ef-4b76-9375-6340072999e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:48:32.352808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee39ff3-75ef-4b76-9375-6340072999e8 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:48:32.357985 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fe4c1-7024-44da-9693-b8dbe8422380 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:48:32.358632 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fe4c1-7024-44da-9693-b8dbe8422380 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:48:32.358945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fe4c1-7024-44da-9693-b8dbe8422380 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:48:32.359225 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fe4c1-7024-44da-9693-b8dbe8422380 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:48:32.367978 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96fe4c1-7024-44da-9693-b8dbe8422380 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:48:32.369592 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96fe4c1-7024-44da-9693-b8dbe8422380 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:48:32.380210 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1413/2866] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:48:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:48:42.163428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=17 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:48:42.375853 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=37 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:48:49.688180 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6a3fe-e7e5-4503-9cce-1f7277483711 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:48:49.696843 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6a3fe-e7e5-4503-9cce-1f7277483711 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:48:49.700764 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e0ea5-391a-434b-a15c-bc6cd81ee347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:48:49.701438 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e0ea5-391a-434b-a15c-bc6cd81ee347 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:48:49.701726 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e0ea5-391a-434b-a15c-bc6cd81ee347 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:48:49.702004 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e0ea5-391a-434b-a15c-bc6cd81ee347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:48:49.710344 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e0ea5-391a-434b-a15c-bc6cd81ee347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:48:49.711770 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e0ea5-391a-434b-a15c-bc6cd81ee347 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:48:49.740031 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1454/2867] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:48:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:48:59.737497 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=13 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:51:27.464744 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fe4c1-7024-44da-9693-b8dbe8422380 service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:51:27.466379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a96fe4c1-7024-44da-9693-b8dbe8422380 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:51:27.544403 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fe4c1-7024-44da-9693-b8dbe8422380 service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:51:27.547913 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8658d-c4a9-4625-ad5a-803a26d71851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:51:27.548551 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8658d-c4a9-4625-ad5a-803a26d71851 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:51:27.548832 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8658d-c4a9-4625-ad5a-803a26d71851 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:51:27.549107 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8658d-c4a9-4625-ad5a-803a26d71851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:51:27.550983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0ba8658d-c4a9-4625-ad5a-803a26d71851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:51:27.640606 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8658d-c4a9-4625-ad5a-803a26d71851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:51:27.642312 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8658d-c4a9-4625-ad5a-803a26d71851 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:51:27.661973 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1414/2868] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:51:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:51:37.659568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=75 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:51:40.971271 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e0ea5-391a-434b-a15c-bc6cd81ee347 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:51:40.981149 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e0ea5-391a-434b-a15c-bc6cd81ee347 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:51:40.984728 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ed9d5-0e6e-43d1-ad8a-dcdfcfce18c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:51:40.985392 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ed9d5-0e6e-43d1-ad8a-dcdfcfce18c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:51:40.985674 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ed9d5-0e6e-43d1-ad8a-dcdfcfce18c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:51:40.985949 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ed9d5-0e6e-43d1-ad8a-dcdfcfce18c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:51:40.987785 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-da9ed9d5-0e6e-43d1-ad8a-dcdfcfce18c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:51:40.997678 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ed9d5-0e6e-43d1-ad8a-dcdfcfce18c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:51:40.999119 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ed9d5-0e6e-43d1-ad8a-dcdfcfce18c2 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:51:41.007127 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1455/2869] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:51:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:51:51.003475 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=6 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:52:22.417569 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8658d-c4a9-4625-ad5a-803a26d71851 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:52:22.427267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8658d-c4a9-4625-ad5a-803a26d71851 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:52:22.430945 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6669660-bafd-4d1b-af61-1c223d951f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:52:22.431590 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6669660-bafd-4d1b-af61-1c223d951f96 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:52:22.431874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6669660-bafd-4d1b-af61-1c223d951f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:52:22.432151 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6669660-bafd-4d1b-af61-1c223d951f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:52:22.434049 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d6669660-bafd-4d1b-af61-1c223d951f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:52:22.448533 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6669660-bafd-4d1b-af61-1c223d951f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:52:22.449977 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6669660-bafd-4d1b-af61-1c223d951f96 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:52:22.461457 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1415/2870] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:52:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:52:32.456702 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:53:23.854703 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ed9d5-0e6e-43d1-ad8a-dcdfcfce18c2 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:53:23.857330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-da9ed9d5-0e6e-43d1-ad8a-dcdfcfce18c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:53:23.876521 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ed9d5-0e6e-43d1-ad8a-dcdfcfce18c2 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:53:23.883428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056d995-e9f4-4c3a-8c85-4ffb63fefd37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:53:23.884077 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056d995-e9f4-4c3a-8c85-4ffb63fefd37 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:53:23.884359 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056d995-e9f4-4c3a-8c85-4ffb63fefd37 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:53:23.884634 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056d995-e9f4-4c3a-8c85-4ffb63fefd37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:53:23.886534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9056d995-e9f4-4c3a-8c85-4ffb63fefd37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:53:23.899668 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056d995-e9f4-4c3a-8c85-4ffb63fefd37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:53:23.901364 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056d995-e9f4-4c3a-8c85-4ffb63fefd37 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:53:23.913318 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1456/2871] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:53:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:53:31.681752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6669660-bafd-4d1b-af61-1c223d951f96 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:53:31.683253 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d6669660-bafd-4d1b-af61-1c223d951f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:53:31.697306 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6669660-bafd-4d1b-af61-1c223d951f96 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:53:31.703273 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e6ce5-eae8-4057-a906-5c340afed925 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:53:31.703909 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e6ce5-eae8-4057-a906-5c340afed925 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:53:31.704189 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e6ce5-eae8-4057-a906-5c340afed925 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:53:31.704465 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e6ce5-eae8-4057-a906-5c340afed925 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:53:31.712927 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e6ce5-eae8-4057-a906-5c340afed925 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:53:31.714511 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e6ce5-eae8-4057-a906-5c340afed925 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:53:31.742221 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1416/2872] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:53:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:53:33.907894 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=6 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:53:35.225978 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056d995-e9f4-4c3a-8c85-4ffb63fefd37 service nova] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:53:35.227641 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-9056d995-e9f4-4c3a-8c85-4ffb63fefd37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:53:35.299328 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056d995-e9f4-4c3a-8c85-4ffb63fefd37 service nova] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:53:35.304970 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52505f-3827-422d-a072-c0114200b24a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:53:35.305655 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52505f-3827-422d-a072-c0114200b24a service placement] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:53:35.305941 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52505f-3827-422d-a072-c0114200b24a service placement] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:53:35.306220 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52505f-3827-422d-a072-c0114200b24a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:53:35.308119 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-3e52505f-3827-422d-a072-c0114200b24a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:53:35.321723 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52505f-3827-422d-a072-c0114200b24a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:53:35.323349 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52505f-3827-422d-a072-c0114200b24a service placement] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:53:35.334591 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1457/2873] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:53:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:53:41.739280 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=13 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:53:45.329356 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=32 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:53:50.345042 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e6ce5-eae8-4057-a906-5c340afed925 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:53:50.356379 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e6ce5-eae8-4057-a906-5c340afed925 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:53:50.361929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c162d0-6521-4ae6-9f36-2aa6672ff94a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:53:50.362568 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c162d0-6521-4ae6-9f36-2aa6672ff94a service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:53:50.362852 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c162d0-6521-4ae6-9f36-2aa6672ff94a service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:53:50.363130 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c162d0-6521-4ae6-9f36-2aa6672ff94a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:53:50.365035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c9c162d0-6521-4ae6-9f36-2aa6672ff94a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:53:50.436553 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c162d0-6521-4ae6-9f36-2aa6672ff94a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:53:50.438166 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c162d0-6521-4ae6-9f36-2aa6672ff94a service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:53:50.448169 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1417/2874] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:53:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:54:00.444313 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=33 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:57:23.227152 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52505f-3827-422d-a072-c0114200b24a service placement] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:57:23.237611 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52505f-3827-422d-a072-c0114200b24a service placement] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:57:23.241235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2edbb-7e21-4d40-874a-5b9c2d185f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:57:23.241870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2edbb-7e21-4d40-874a-5b9c2d185f65 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:57:23.242151 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2edbb-7e21-4d40-874a-5b9c2d185f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:57:23.242424 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2edbb-7e21-4d40-874a-5b9c2d185f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:57:23.253841 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2edbb-7e21-4d40-874a-5b9c2d185f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:57:23.255291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2edbb-7e21-4d40-874a-5b9c2d185f65 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:57:23.267547 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1458/2875] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:57:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:57:33.262162 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=6 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:58:41.246242 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c162d0-6521-4ae6-9f36-2aa6672ff94a service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:58:41.247808 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c9c162d0-6521-4ae6-9f36-2aa6672ff94a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:58:41.331261 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c162d0-6521-4ae6-9f36-2aa6672ff94a service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:58:41.336383 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b429a-20b6-40ce-8dc6-2b721bd4af12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:58:41.337348 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b429a-20b6-40ce-8dc6-2b721bd4af12 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:58:41.337640 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b429a-20b6-40ce-8dc6-2b721bd4af12 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:58:41.337923 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b429a-20b6-40ce-8dc6-2b721bd4af12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:58:41.346651 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b429a-20b6-40ce-8dc6-2b721bd4af12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:58:41.348223 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b429a-20b6-40ce-8dc6-2b721bd4af12 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:58:41.358293 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1418/2876] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:58:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:58:50.225750 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2edbb-7e21-4d40-874a-5b9c2d185f65 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:58:50.271071 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2edbb-7e21-4d40-874a-5b9c2d185f65 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:58:50.276824 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b95a9-41f7-4169-b734-66726c36b1a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:58:50.277494 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b95a9-41f7-4169-b734-66726c36b1a0 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:58:50.277777 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b95a9-41f7-4169-b734-66726c36b1a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:58:50.278053 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b95a9-41f7-4169-b734-66726c36b1a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:58:50.285870 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b95a9-41f7-4169-b734-66726c36b1a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:58:50.287450 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b95a9-41f7-4169-b734-66726c36b1a0 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:58:50.314243 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1459/2877] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:58:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:58:51.354007 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=37 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:59:00.311437 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=27 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:59:22.272265 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56104a-0e0f-4a73-a88c-7ff596e56035 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:22.272738 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56104a-0e0f-4a73-a88c-7ff596e56035 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:22.272852 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56104a-0e0f-4a73-a88c-7ff596e56035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:22.329474 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ad56104a-0e0f-4a73-a88c-7ff596e56035 None None] MFA Rules not processed for user `60f89f6042664f91b2829d516ed7e844`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:59:22.330665 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ad56104a-0e0f-4a73-a88c-7ff596e56035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:22.394400 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1419/2878] 10.222.0.42 () {58 vars in 967 bytes} [Tue Jul 19 15:59:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:59:22.434449 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b95a9-41f7-4169-b734-66726c36b1a0 service nova] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:22.436031 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-819b95a9-41f7-4169-b734-66726c36b1a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:22.526280 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b95a9-41f7-4169-b734-66726c36b1a0 service nova] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:22.531429 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95cbc0-e533-4aca-b5be-75a387c26922 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96faaac55bc5408685c68a6177d91971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:22.532079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95cbc0-e533-4aca-b5be-75a387c26922 service swift] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:22.532366 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95cbc0-e533-4aca-b5be-75a387c26922 service swift] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:22.532647 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95cbc0-e533-4aca-b5be-75a387c26922 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:22.541474 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95cbc0-e533-4aca-b5be-75a387c26922 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:22.543071 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95cbc0-e533-4aca-b5be-75a387c26922 service swift] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:22.551145 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1460/2879] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Jul 19 15:59:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 15:59:22.988990 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56104a-0e0f-4a73-a88c-7ff596e56035 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:22.990700 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ad56104a-0e0f-4a73-a88c-7ff596e56035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:23.071779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56104a-0e0f-4a73-a88c-7ff596e56035 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:23.075798 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68517a4e-09d2-4170-b1cf-0c5108c6be48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:23.076581 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68517a4e-09d2-4170-b1cf-0c5108c6be48 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:23.076730 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68517a4e-09d2-4170-b1cf-0c5108c6be48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:23.077010 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68517a4e-09d2-4170-b1cf-0c5108c6be48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:23.078955 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-68517a4e-09d2-4170-b1cf-0c5108c6be48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:23.095953 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95cbc0-e533-4aca-b5be-75a387c26922 service swift] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:23.105886 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95cbc0-e533-4aca-b5be-75a387c26922 service swift] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:23.109268 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9d65e-88ac-4abf-9377-341bf69d24a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:23.109911 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9d65e-88ac-4abf-9377-341bf69d24a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:23.110194 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9d65e-88ac-4abf-9377-341bf69d24a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:23.110470 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9d65e-88ac-4abf-9377-341bf69d24a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:23.112288 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-aee9d65e-88ac-4abf-9377-341bf69d24a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:23.144109 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:59:23.158417 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68517a4e-09d2-4170-b1cf-0c5108c6be48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:23.159864 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68517a4e-09d2-4170-b1cf-0c5108c6be48 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:23.169043 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1420/2880] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:59:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:59:23.200901 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9d65e-88ac-4abf-9377-341bf69d24a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:23.202490 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9d65e-88ac-4abf-9377-341bf69d24a3 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:23.222313 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1461/2881] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:59:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 15:59:26.273281 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68517a4e-09d2-4170-b1cf-0c5108c6be48 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:26.286401 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68517a4e-09d2-4170-b1cf-0c5108c6be48 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:26.291686 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f133e0f-5f4f-4e5e-9309-1ced3823f393 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:26.292338 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f133e0f-5f4f-4e5e-9309-1ced3823f393 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:26.292623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f133e0f-5f4f-4e5e-9309-1ced3823f393 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:26.292900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f133e0f-5f4f-4e5e-9309-1ced3823f393 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:26.294713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2f133e0f-5f4f-4e5e-9309-1ced3823f393 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:26.309983 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f133e0f-5f4f-4e5e-9309-1ced3823f393 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:26.313193 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f133e0f-5f4f-4e5e-9309-1ced3823f393 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:26.334765 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1421/2882] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:59:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:59:29.656228 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9d65e-88ac-4abf-9377-341bf69d24a3 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:29.668259 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9d65e-88ac-4abf-9377-341bf69d24a3 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:29.673257 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321050f-922a-4e06-82f0-585fa886d062 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:29.673915 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321050f-922a-4e06-82f0-585fa886d062 service placement] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:29.674198 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321050f-922a-4e06-82f0-585fa886d062 service placement] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:29.674479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321050f-922a-4e06-82f0-585fa886d062 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:29.676291 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-2321050f-922a-4e06-82f0-585fa886d062 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:29.688857 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321050f-922a-4e06-82f0-585fa886d062 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:29.690478 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321050f-922a-4e06-82f0-585fa886d062 service placement] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:29.700937 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1462/2883] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:59:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:59:29.719944 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f133e0f-5f4f-4e5e-9309-1ced3823f393 service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:29.722086 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2f133e0f-5f4f-4e5e-9309-1ced3823f393 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:29.740380 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f133e0f-5f4f-4e5e-9309-1ced3823f393 service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:29.745947 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab26b3-0c7b-4365-9d73-52845988ee8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:29.746807 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab26b3-0c7b-4365-9d73-52845988ee8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:29.747085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab26b3-0c7b-4365-9d73-52845988ee8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:29.747354 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab26b3-0c7b-4365-9d73-52845988ee8e admin admin] PATH_INFO: `/v3/users/15af8262a49946739a79f12ec7f8b15d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:29.748575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab26b3-0c7b-4365-9d73-52845988ee8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=15af8262a49946739a79f12ec7f8b15d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:29.750844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab26b3-0c7b-4365-9d73-52845988ee8e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:29.797232 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1bab26b3-0c7b-4365-9d73-52845988ee8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15af8262a49946739a79f12ec7f8b15d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:29.809315 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1bab26b3-0c7b-4365-9d73-52845988ee8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15af8262a49946739a79f12ec7f8b15d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:29.809959 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1bab26b3-0c7b-4365-9d73-52845988ee8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15af8262a49946739a79f12ec7f8b15d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:29.810559 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1bab26b3-0c7b-4365-9d73-52845988ee8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15af8262a49946739a79f12ec7f8b15d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:29.817267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1bab26b3-0c7b-4365-9d73-52845988ee8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15af8262a49946739a79f12ec7f8b15d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:29.829723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1422/2884] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:59:29 2022] DELETE /identity/v3/users/15af8262a49946739a79f12ec7f8b15d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:59:29.997184 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321050f-922a-4e06-82f0-585fa886d062 service placement] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:30.008270 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321050f-922a-4e06-82f0-585fa886d062 service placement] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:30.027526 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=100 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:59:30.067829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d53dba-c9b6-47ea-aa1f-601a6b27be49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:30.068893 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d53dba-c9b6-47ea-aa1f-601a6b27be49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:30.069222 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d53dba-c9b6-47ea-aa1f-601a6b27be49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:30.069502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d53dba-c9b6-47ea-aa1f-601a6b27be49 admin admin] PATH_INFO: `/v3/projects/05857a00bdb14c54800b5e0f55ed27fc` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:30.070802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d53dba-c9b6-47ea-aa1f-601a6b27be49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05857a00bdb14c54800b5e0f55ed27fc)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:30.072812 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d53dba-c9b6-47ea-aa1f-601a6b27be49 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:30.119369 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-78d53dba-c9b6-47ea-aa1f-601a6b27be49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05857a00bdb14c54800b5e0f55ed27fc', 'request_id': 'req-78d53dba-c9b6-47ea-aa1f-601a6b27be49'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:30.124740 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-78d53dba-c9b6-47ea-aa1f-601a6b27be49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05857a00bdb14c54800b5e0f55ed27fc', 'request_id': 'req-78d53dba-c9b6-47ea-aa1f-601a6b27be49'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:30.130834 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-78d53dba-c9b6-47ea-aa1f-601a6b27be49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05857a00bdb14c54800b5e0f55ed27fc', 'request_id': 'req-78d53dba-c9b6-47ea-aa1f-601a6b27be49'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:30.142883 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-78d53dba-c9b6-47ea-aa1f-601a6b27be49 admin admin] The token cache is being invalidate because project 05857a00bdb14c54800b5e0f55ed27fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:59:30.143229 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-78d53dba-c9b6-47ea-aa1f-601a6b27be49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:30.143844 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-78d53dba-c9b6-47ea-aa1f-601a6b27be49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:30.145394 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1463/2885] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:59:29 2022] DELETE /identity/v3/projects/05857a00bdb14c54800b5e0f55ed27fc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:59:30.150072 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab26b3-0c7b-4365-9d73-52845988ee8e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:30.151454 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1bab26b3-0c7b-4365-9d73-52845988ee8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:30.165058 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab26b3-0c7b-4365-9d73-52845988ee8e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:30.221203 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d31bf-22fb-4ffc-a679-852e2d69f6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:30.222082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d31bf-22fb-4ffc-a679-852e2d69f6a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:30.222370 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d31bf-22fb-4ffc-a679-852e2d69f6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:30.222652 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d31bf-22fb-4ffc-a679-852e2d69f6a8 admin admin] PATH_INFO: `/v3/users/155dd98501894a1d856de203dfc6689a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:30.235937 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d31bf-22fb-4ffc-a679-852e2d69f6a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=155dd98501894a1d856de203dfc6689a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:30.237905 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d31bf-22fb-4ffc-a679-852e2d69f6a8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:30.276752 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:59:30.287289 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e81d31bf-22fb-4ffc-a679-852e2d69f6a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '155dd98501894a1d856de203dfc6689a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:30.299381 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e81d31bf-22fb-4ffc-a679-852e2d69f6a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '155dd98501894a1d856de203dfc6689a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:30.300039 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e81d31bf-22fb-4ffc-a679-852e2d69f6a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '155dd98501894a1d856de203dfc6689a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:30.300635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e81d31bf-22fb-4ffc-a679-852e2d69f6a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '155dd98501894a1d856de203dfc6689a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:30.307623 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e81d31bf-22fb-4ffc-a679-852e2d69f6a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '155dd98501894a1d856de203dfc6689a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:30.322057 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1423/2886] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 15:59:30 2022] DELETE /identity/v3/users/155dd98501894a1d856de203dfc6689a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:59:30.491670 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d53dba-c9b6-47ea-aa1f-601a6b27be49 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:30.493904 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-78d53dba-c9b6-47ea-aa1f-601a6b27be49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:30.506965 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d53dba-c9b6-47ea-aa1f-601a6b27be49 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:30.564695 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9ce78-7597-40c0-956f-1771bde7770c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:30.565594 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9ce78-7597-40c0-956f-1771bde7770c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:30.565879 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9ce78-7597-40c0-956f-1771bde7770c admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:30.566158 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9ce78-7597-40c0-956f-1771bde7770c admin admin] PATH_INFO: `/v3/projects/2fa7bda25eec409f88dc10e649241132` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:30.579461 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9ce78-7597-40c0-956f-1771bde7770c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fa7bda25eec409f88dc10e649241132)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:30.581357 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9ce78-7597-40c0-956f-1771bde7770c admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:30.624563 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:59:30.627748 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c7e9ce78-7597-40c0-956f-1771bde7770c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fa7bda25eec409f88dc10e649241132', 'request_id': 'req-c7e9ce78-7597-40c0-956f-1771bde7770c'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:30.634279 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c7e9ce78-7597-40c0-956f-1771bde7770c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fa7bda25eec409f88dc10e649241132', 'request_id': 'req-c7e9ce78-7597-40c0-956f-1771bde7770c'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:30.641112 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c7e9ce78-7597-40c0-956f-1771bde7770c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fa7bda25eec409f88dc10e649241132', 'request_id': 'req-c7e9ce78-7597-40c0-956f-1771bde7770c'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:30.654297 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c7e9ce78-7597-40c0-956f-1771bde7770c admin admin] The token cache is being invalidate because project 2fa7bda25eec409f88dc10e649241132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80271) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 15:59:30.654584 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c7e9ce78-7597-40c0-956f-1771bde7770c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:30.655198 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-c7e9ce78-7597-40c0-956f-1771bde7770c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 15:59:30.656891 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1464/2887] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 15:59:30 2022] DELETE /identity/v3/projects/2fa7bda25eec409f88dc10e649241132 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:59:30.661840 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4c4c9-fbad-4039-8bc8-f762b69bfa14 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:30.662136 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4c4c9-fbad-4039-8bc8-f762b69bfa14 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:30.662412 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4c4c9-fbad-4039-8bc8-f762b69bfa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:30.718101 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e8a4c4c9-fbad-4039-8bc8-f762b69bfa14 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:59:30.719304 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e8a4c4c9-fbad-4039-8bc8-f762b69bfa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:30.778580 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1424/2888] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:59:30.786409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85beaf9c-aaff-4bc6-ad85-3dbe8e12e26a None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:30.786704 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85beaf9c-aaff-4bc6-ad85-3dbe8e12e26a None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:30.786986 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85beaf9c-aaff-4bc6-ad85-3dbe8e12e26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:30.805791 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-85beaf9c-aaff-4bc6-ad85-3dbe8e12e26a None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:59:30.807010 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-85beaf9c-aaff-4bc6-ad85-3dbe8e12e26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:30.816371 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1465/2889] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:59:30.820729 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4c4c9-fbad-4039-8bc8-f762b69bfa14 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:30.830764 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4c4c9-fbad-4039-8bc8-f762b69bfa14 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:30.835380 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da6f8c-5a71-4d5d-bd59-63c7d3e55991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:30.836038 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da6f8c-5a71-4d5d-bd59-63c7d3e55991 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:30.836843 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da6f8c-5a71-4d5d-bd59-63c7d3e55991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:30.837102 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da6f8c-5a71-4d5d-bd59-63c7d3e55991 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:30.838027 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da6f8c-5a71-4d5d-bd59-63c7d3e55991 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 15:59:30.838309 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da6f8c-5a71-4d5d-bd59-63c7d3e55991 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:30.839970 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da6f8c-5a71-4d5d-bd59-63c7d3e55991 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:30.855575 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1425/2890] 10.222.0.42 () {58 vars in 1152 bytes} [Tue Jul 19 15:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 15:59:30.857504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85beaf9c-aaff-4bc6-ad85-3dbe8e12e26a None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:30.867428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85beaf9c-aaff-4bc6-ad85-3dbe8e12e26a None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:30.872539 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2148005-2a3c-4efd-bc2a-d4178f5f7dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:30.873239 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2148005-2a3c-4efd-bc2a-d4178f5f7dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:30.873524 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2148005-2a3c-4efd-bc2a-d4178f5f7dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:30.873802 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2148005-2a3c-4efd-bc2a-d4178f5f7dc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:30.874504 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2148005-2a3c-4efd-bc2a-d4178f5f7dc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:30.876441 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2148005-2a3c-4efd-bc2a-d4178f5f7dc3 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:30.912946 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1466/2891] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:59:30 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:59:30.917779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da6f8c-5a71-4d5d-bd59-63c7d3e55991 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:30.927755 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da6f8c-5a71-4d5d-bd59-63c7d3e55991 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:30.984219 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7958930-4219-4c6c-83c9-87c68c719d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:30.984890 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7958930-4219-4c6c-83c9-87c68c719d4c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:30.985205 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7958930-4219-4c6c-83c9-87c68c719d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:30.985486 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7958930-4219-4c6c-83c9-87c68c719d4c admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:30.986188 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7958930-4219-4c6c-83c9-87c68c719d4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:30.988122 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7958930-4219-4c6c-83c9-87c68c719d4c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:31.002787 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:59:31.009343 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1426/2892] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:59:30 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 15:59:31.013531 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2148005-2a3c-4efd-bc2a-d4178f5f7dc3 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:31.024428 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2148005-2a3c-4efd-bc2a-d4178f5f7dc3 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:31.029088 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4130d-d159-4ab5-b096-a8df4adf7849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:31.029773 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4130d-d159-4ab5-b096-a8df4adf7849 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:31.030060 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4130d-d159-4ab5-b096-a8df4adf7849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:31.030341 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4130d-d159-4ab5-b096-a8df4adf7849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:31.031084 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4130d-d159-4ab5-b096-a8df4adf7849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:31.032689 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4130d-d159-4ab5-b096-a8df4adf7849 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:31.042596 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1467/2893] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:59:31.046498 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7958930-4219-4c6c-83c9-87c68c719d4c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:31.056567 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7958930-4219-4c6c-83c9-87c68c719d4c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:31.061159 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531cdb1-1527-454f-bb26-8439c49a69ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:31.061896 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531cdb1-1527-454f-bb26-8439c49a69ed admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:31.062204 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531cdb1-1527-454f-bb26-8439c49a69ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:31.062483 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531cdb1-1527-454f-bb26-8439c49a69ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:31.063267 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531cdb1-1527-454f-bb26-8439c49a69ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:31.064823 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531cdb1-1527-454f-bb26-8439c49a69ed admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:31.075760 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1427/2894] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:59:31.079707 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4130d-d159-4ab5-b096-a8df4adf7849 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:31.090376 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4130d-d159-4ab5-b096-a8df4adf7849 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:31.094961 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b877120-0383-44ae-909b-e553c750e782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:31.095828 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b877120-0383-44ae-909b-e553c750e782 admin admin] REQUEST_METHOD: `PUT` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:31.096108 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b877120-0383-44ae-909b-e553c750e782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:31.096385 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b877120-0383-44ae-909b-e553c750e782 admin admin] PATH_INFO: `/v3/projects/ae073bb1e3214abaabdf711901bfa192/users/f1d44fdb8f544694919f673949a9f182/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:31.110881 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b877120-0383-44ae-909b-e553c750e782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae073bb1e3214abaabdf711901bfa192, user_id=f1d44fdb8f544694919f673949a9f182, role_id=d6e9d1df8d5b44c2a000a1ca9a35b04f)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:31.113582 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b877120-0383-44ae-909b-e553c750e782 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:31.126408 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1468/2895] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:59:31 2022] PUT /identity/v3/projects/ae073bb1e3214abaabdf711901bfa192/users/f1d44fdb8f544694919f673949a9f182/roles/d6e9d1df8d5b44c2a000a1ca9a35b04f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:59:31.131447 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730beefb-cd09-49dd-9293-fa26529bc905 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:31.131733 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730beefb-cd09-49dd-9293-fa26529bc905 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:31.132009 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730beefb-cd09-49dd-9293-fa26529bc905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:31.150864 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-730beefb-cd09-49dd-9293-fa26529bc905 None None] MFA Rules not processed for user `669959f6d82b46b1b551b6c19bf21c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:59:31.152052 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-730beefb-cd09-49dd-9293-fa26529bc905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:31.208327 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1428/2896] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:59:31.218906 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b877120-0383-44ae-909b-e553c750e782 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:31.220577 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-1b877120-0383-44ae-909b-e553c750e782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:31.278464 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b877120-0383-44ae-909b-e553c750e782 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:31.282403 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad173-5859-4a74-bfc8-b4e4f882328a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:31.283056 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad173-5859-4a74-bfc8-b4e4f882328a service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:31.283343 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad173-5859-4a74-bfc8-b4e4f882328a service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:31.283623 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad173-5859-4a74-bfc8-b4e4f882328a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:31.293055 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ad173-5859-4a74-bfc8-b4e4f882328a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:31.294525 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ad173-5859-4a74-bfc8-b4e4f882328a service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:31.303872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1469/2897] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:59:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:59:35.020449 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50caeb86-8f32-4f98-95db-85fa8cdb19b0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:35.020950 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50caeb86-8f32-4f98-95db-85fa8cdb19b0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:35.021354 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50caeb86-8f32-4f98-95db-85fa8cdb19b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:35.051482 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-50caeb86-8f32-4f98-95db-85fa8cdb19b0 None None] MFA Rules not processed for user `f1d44fdb8f544694919f673949a9f182`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:59:35.052722 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-50caeb86-8f32-4f98-95db-85fa8cdb19b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:35.132262 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1429/2898] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:59:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:59:35.137860 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad173-5859-4a74-bfc8-b4e4f882328a service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:35.152438 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad173-5859-4a74-bfc8-b4e4f882328a service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:35.157423 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4c7f6-f69c-4b36-8938-c2f0c48efd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:35.158098 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4c7f6-f69c-4b36-8938-c2f0c48efd22 admin admin] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:35.158384 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4c7f6-f69c-4b36-8938-c2f0c48efd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:35.158664 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4c7f6-f69c-4b36-8938-c2f0c48efd22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:35.159571 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4c7f6-f69c-4b36-8938-c2f0c48efd22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:35.161560 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4c7f6-f69c-4b36-8938-c2f0c48efd22 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:35.200783 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1470/2899] 10.222.0.42 () {60 vars in 1152 bytes} [Tue Jul 19 15:59:35 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:59:35.202661 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50caeb86-8f32-4f98-95db-85fa8cdb19b0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:35.216615 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50caeb86-8f32-4f98-95db-85fa8cdb19b0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:35.293104 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15b92b-1158-4e5e-9872-4868860e2d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:35.293834 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15b92b-1158-4e5e-9872-4868860e2d52 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:35.294116 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15b92b-1158-4e5e-9872-4868860e2d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:35.294548 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15b92b-1158-4e5e-9872-4868860e2d52 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:35.295252 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15b92b-1158-4e5e-9872-4868860e2d52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:35.297439 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15b92b-1158-4e5e-9872-4868860e2d52 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:35.328634 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1430/2900] 10.222.0.42 () {60 vars in 1143 bytes} [Tue Jul 19 15:59:35 2022] POST /identity/v3/users => generated 295 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 15:59:35.329648 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4c7f6-f69c-4b36-8938-c2f0c48efd22 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:35.341781 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4c7f6-f69c-4b36-8938-c2f0c48efd22 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:35.346453 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd7c82-d272-42b9-8d8d-15eef083d858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:35.347140 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd7c82-d272-42b9-8d8d-15eef083d858 admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:35.347409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd7c82-d272-42b9-8d8d-15eef083d858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:35.347712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd7c82-d272-42b9-8d8d-15eef083d858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:35.348458 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd7c82-d272-42b9-8d8d-15eef083d858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:35.350113 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd7c82-d272-42b9-8d8d-15eef083d858 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:35.361940 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1471/2901] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:59:35.366512 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15b92b-1158-4e5e-9872-4868860e2d52 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:35.379200 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15b92b-1158-4e5e-9872-4868860e2d52 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:35.385412 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8b92b-804b-4f4e-bb64-78e32131f9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:35.386714 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8b92b-804b-4f4e-bb64-78e32131f9ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:35.387170 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8b92b-804b-4f4e-bb64-78e32131f9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:35.387455 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8b92b-804b-4f4e-bb64-78e32131f9ab admin admin] PATH_INFO: `/v3/projects/3debca74dc904d3a8a26677c0ec27d25/users/c776745776ee433caa4218d75aa966ce/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:35.390969 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:59:35.406508 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8b92b-804b-4f4e-bb64-78e32131f9ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3debca74dc904d3a8a26677c0ec27d25, user_id=c776745776ee433caa4218d75aa966ce, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:35.409224 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8b92b-804b-4f4e-bb64-78e32131f9ab admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:35.423705 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1431/2902] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:59:35 2022] PUT /identity/v3/projects/3debca74dc904d3a8a26677c0ec27d25/users/c776745776ee433caa4218d75aa966ce/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 15:59:35.427616 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd7c82-d272-42b9-8d8d-15eef083d858 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:35.438249 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd7c82-d272-42b9-8d8d-15eef083d858 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:35.466336 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:59:35.497097 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c8193-c8fd-4107-b74b-3c3591b6ec9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:35.497792 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c8193-c8fd-4107-b74b-3c3591b6ec9e admin admin] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:35.498078 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c8193-c8fd-4107-b74b-3c3591b6ec9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:35.498353 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c8193-c8fd-4107-b74b-3c3591b6ec9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:35.499079 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c8193-c8fd-4107-b74b-3c3591b6ec9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:35.500675 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c8193-c8fd-4107-b74b-3c3591b6ec9e admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:35.510662 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1472/2903] 10.222.0.42 () {58 vars in 1121 bytes} [Tue Jul 19 15:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 15:59:35.515240 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8b92b-804b-4f4e-bb64-78e32131f9ab admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:35.525929 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8b92b-804b-4f4e-bb64-78e32131f9ab admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:35.530473 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048e757-5c99-4beb-88c2-be48906826ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:35.531341 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048e757-5c99-4beb-88c2-be48906826ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:35.531618 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048e757-5c99-4beb-88c2-be48906826ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:35.531898 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048e757-5c99-4beb-88c2-be48906826ee admin admin] PATH_INFO: `/v3/projects/3debca74dc904d3a8a26677c0ec27d25/users/c776745776ee433caa4218d75aa966ce/roles/113045fe7ac44a0998b8ec6fe410d564` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:35.534140 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048e757-5c99-4beb-88c2-be48906826ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=3debca74dc904d3a8a26677c0ec27d25, user_id=c776745776ee433caa4218d75aa966ce, role_id=113045fe7ac44a0998b8ec6fe410d564)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:35.536723 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048e757-5c99-4beb-88c2-be48906826ee admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:35.547528 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1432/2904] 10.222.0.42 () {60 vars in 1482 bytes} [Tue Jul 19 15:59:35 2022] PUT /identity/v3/projects/3debca74dc904d3a8a26677c0ec27d25/users/c776745776ee433caa4218d75aa966ce/roles/113045fe7ac44a0998b8ec6fe410d564 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 15:59:40.190502 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bfb0e-1c26-4b6b-a920-344a81c71570 None None] REQUEST_METHOD: `POST` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:40.191069 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bfb0e-1c26-4b6b-a920-344a81c71570 None None] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:40.191197 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bfb0e-1c26-4b6b-a920-344a81c71570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:40.221492 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.auth.core [None req-687bfb0e-1c26-4b6b-a920-344a81c71570 None None] MFA Rules not processed for user `c776745776ee433caa4218d75aa966ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80271) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 15:59:40.222731 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-687bfb0e-1c26-4b6b-a920-344a81c71570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:40.302152 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1473/2905] 10.222.0.42 () {58 vars in 957 bytes} [Tue Jul 19 15:59:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 15:59:40.321725 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048e757-5c99-4beb-88c2-be48906826ee admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 15:59:40.324164 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b048e757-5c99-4beb-88c2-be48906826ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 15:59:40.403115 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048e757-5c99-4beb-88c2-be48906826ee admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 15:59:40.408731 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a50717-5788-4cb7-a7d1-10ce2cf486c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 15:59:40.409417 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a50717-5788-4cb7-a7d1-10ce2cf486c9 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 15:59:40.409704 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a50717-5788-4cb7-a7d1-10ce2cf486c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 15:59:40.409980 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a50717-5788-4cb7-a7d1-10ce2cf486c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 15:59:40.469085 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a50717-5788-4cb7-a7d1-10ce2cf486c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 15:59:40.470709 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a50717-5788-4cb7-a7d1-10ce2cf486c9 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 15:59:40.480739 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1433/2906] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 15:59:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 15:59:50.298522 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=49 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 15:59:50.475874 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=67,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 16:00:47.972817 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bfb0e-1c26-4b6b-a920-344a81c71570 None None] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 16:00:48.042065 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bfb0e-1c26-4b6b-a920-344a81c71570 None None] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 16:00:48.047529 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa12c8-4e55-4f59-b242-a071088d6eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 16:00:48.048572 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa12c8-4e55-4f59-b242-a071088d6eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 16:00:48.048855 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa12c8-4e55-4f59-b242-a071088d6eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 16:00:48.049130 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa12c8-4e55-4f59-b242-a071088d6eb2 admin admin] PATH_INFO: `/v3/users/f1d44fdb8f544694919f673949a9f182` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 16:00:48.050368 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa12c8-4e55-4f59-b242-a071088d6eb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1d44fdb8f544694919f673949a9f182)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 16:00:48.052312 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa12c8-4e55-4f59-b242-a071088d6eb2 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 16:00:48.104258 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6faa12c8-4e55-4f59-b242-a071088d6eb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1d44fdb8f544694919f673949a9f182'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.116567 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6faa12c8-4e55-4f59-b242-a071088d6eb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1d44fdb8f544694919f673949a9f182'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.117251 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6faa12c8-4e55-4f59-b242-a071088d6eb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1d44fdb8f544694919f673949a9f182'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.124938 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6faa12c8-4e55-4f59-b242-a071088d6eb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1d44fdb8f544694919f673949a9f182'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.125663 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-6faa12c8-4e55-4f59-b242-a071088d6eb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1d44fdb8f544694919f673949a9f182'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.138721 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1474/2907] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 16:00:47 2022] DELETE /identity/v3/users/f1d44fdb8f544694919f673949a9f182 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 16:00:48.313004 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a50717-5788-4cb7-a7d1-10ce2cf486c9 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 16:00:48.330298 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a50717-5788-4cb7-a7d1-10ce2cf486c9 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 16:00:48.393436 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec26fa-6018-43af-8a94-e6245e504d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 16:00:48.394228 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec26fa-6018-43af-8a94-e6245e504d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 16:00:48.394511 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec26fa-6018-43af-8a94-e6245e504d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 16:00:48.394786 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec26fa-6018-43af-8a94-e6245e504d61 admin admin] PATH_INFO: `/v3/projects/ae073bb1e3214abaabdf711901bfa192` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 16:00:48.396000 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec26fa-6018-43af-8a94-e6245e504d61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae073bb1e3214abaabdf711901bfa192)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 16:00:48.397949 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec26fa-6018-43af-8a94-e6245e504d61 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 16:00:48.444811 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b7ec26fa-6018-43af-8a94-e6245e504d61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae073bb1e3214abaabdf711901bfa192', 'request_id': 'req-b7ec26fa-6018-43af-8a94-e6245e504d61'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.451958 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b7ec26fa-6018-43af-8a94-e6245e504d61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae073bb1e3214abaabdf711901bfa192', 'request_id': 'req-b7ec26fa-6018-43af-8a94-e6245e504d61'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.457863 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b7ec26fa-6018-43af-8a94-e6245e504d61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae073bb1e3214abaabdf711901bfa192', 'request_id': 'req-b7ec26fa-6018-43af-8a94-e6245e504d61'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.469750 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b7ec26fa-6018-43af-8a94-e6245e504d61 admin admin] The token cache is being invalidate because project ae073bb1e3214abaabdf711901bfa192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 16:00:48.470045 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b7ec26fa-6018-43af-8a94-e6245e504d61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.470695 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b7ec26fa-6018-43af-8a94-e6245e504d61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.472238 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1434/2908] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 16:00:48 2022] DELETE /identity/v3/projects/ae073bb1e3214abaabdf711901bfa192 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 16:00:48.476231 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa12c8-4e55-4f59-b242-a071088d6eb2 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 16:00:48.477656 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-6faa12c8-4e55-4f59-b242-a071088d6eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:00:48.491294 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa12c8-4e55-4f59-b242-a071088d6eb2 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 16:00:48.547536 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689b605-928b-4917-90ea-d7195edbfbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 16:00:48.548402 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689b605-928b-4917-90ea-d7195edbfbd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 16:00:48.548684 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689b605-928b-4917-90ea-d7195edbfbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 16:00:48.548960 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689b605-928b-4917-90ea-d7195edbfbd4 admin admin] PATH_INFO: `/v3/users/c776745776ee433caa4218d75aa966ce` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 16:00:48.550195 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689b605-928b-4917-90ea-d7195edbfbd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c776745776ee433caa4218d75aa966ce)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 16:00:48.552120 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689b605-928b-4917-90ea-d7195edbfbd4 admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 16:00:48.573163 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 16:00:48.599043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5689b605-928b-4917-90ea-d7195edbfbd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c776745776ee433caa4218d75aa966ce'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.611409 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5689b605-928b-4917-90ea-d7195edbfbd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c776745776ee433caa4218d75aa966ce'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.612059 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5689b605-928b-4917-90ea-d7195edbfbd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c776745776ee433caa4218d75aa966ce'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.619218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5689b605-928b-4917-90ea-d7195edbfbd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c776745776ee433caa4218d75aa966ce'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.619872 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-5689b605-928b-4917-90ea-d7195edbfbd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c776745776ee433caa4218d75aa966ce'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.632167 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1475/2909] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 16:00:48 2022] DELETE /identity/v3/users/c776745776ee433caa4218d75aa966ce => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 16:00:48.805821 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec26fa-6018-43af-8a94-e6245e504d61 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 16:00:48.807268 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b7ec26fa-6018-43af-8a94-e6245e504d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:00:48.820295 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec26fa-6018-43af-8a94-e6245e504d61 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 16:00:48.891035 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 16:00:48.891900 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 16:00:48.892181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 16:00:48.892456 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621 admin admin] PATH_INFO: `/v3/projects/3debca74dc904d3a8a26677c0ec27d25` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 16:00:48.893658 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3debca74dc904d3a8a26677c0ec27d25)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 16:00:48.895555 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 16:00:48.931778 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 16:00:48.945858 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3debca74dc904d3a8a26677c0ec27d25', 'request_id': 'req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.954518 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3debca74dc904d3a8a26677c0ec27d25', 'request_id': 'req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.961690 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3debca74dc904d3a8a26677c0ec27d25', 'request_id': 'req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.973860 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621 admin admin] The token cache is being invalidate because project 3debca74dc904d3a8a26677c0ec27d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 16:00:48.974149 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.974788 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:00:48.976311 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1435/2910] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 16:00:48 2022] DELETE /identity/v3/projects/3debca74dc904d3a8a26677c0ec27d25 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 16:00:58.628737 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 16:00:58.970881 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 16:01:24.602298 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689b605-928b-4917-90ea-d7195edbfbd4 admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 16:01:24.604173 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5689b605-928b-4917-90ea-d7195edbfbd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:01:24.684629 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689b605-928b-4917-90ea-d7195edbfbd4 admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 16:01:24.751887 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfe65c-6f39-45e0-9806-30678ba9ca59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff6a2d4bc734904b2615608a86b097d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 16:01:24.752548 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfe65c-6f39-45e0-9806-30678ba9ca59 service nova] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 16:01:24.752829 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfe65c-6f39-45e0-9806-30678ba9ca59 service nova] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 16:01:24.753105 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfe65c-6f39-45e0-9806-30678ba9ca59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 16:01:24.755036 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e4cfe65c-6f39-45e0-9806-30678ba9ca59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:01:24.768479 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfe65c-6f39-45e0-9806-30678ba9ca59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 16:01:24.770124 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfe65c-6f39-45e0-9806-30678ba9ca59 service nova] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 16:01:24.781767 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1476/2911] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 16:01:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 16:01:34.776534 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=62 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 16:03:46.165425 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 16:03:46.166793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:03:46.236376 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f058ed-bf5c-40c9-a8bf-ab2dc0f59621 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 16:03:46.242231 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de6146-0e3c-4470-a7fb-2422d4ac650d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 16:03:46.242895 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de6146-0e3c-4470-a7fb-2422d4ac650d service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 16:03:46.243181 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de6146-0e3c-4470-a7fb-2422d4ac650d service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 16:03:46.243463 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de6146-0e3c-4470-a7fb-2422d4ac650d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 16:03:46.245297 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a1de6146-0e3c-4470-a7fb-2422d4ac650d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:03:46.257754 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de6146-0e3c-4470-a7fb-2422d4ac650d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 16:03:46.259336 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de6146-0e3c-4470-a7fb-2422d4ac650d service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 16:03:46.269362 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1436/2912] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 16:03:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 16:03:56.265023 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=32 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 16:04:23.752981 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfe65c-6f39-45e0-9806-30678ba9ca59 service nova] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 16:04:23.754439 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-e4cfe65c-6f39-45e0-9806-30678ba9ca59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:04:23.810039 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfe65c-6f39-45e0-9806-30678ba9ca59 service nova] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 16:04:23.813693 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceef563-9f4d-4e5b-91ef-43cb72f950c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 16:04:23.814330 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceef563-9f4d-4e5b-91ef-43cb72f950c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 16:04:23.814612 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceef563-9f4d-4e5b-91ef-43cb72f950c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 16:04:23.814885 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceef563-9f4d-4e5b-91ef-43cb72f950c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 16:04:23.816611 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5ceef563-9f4d-4e5b-91ef-43cb72f950c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:04:23.883146 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceef563-9f4d-4e5b-91ef-43cb72f950c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 16:04:23.884570 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceef563-9f4d-4e5b-91ef-43cb72f950c9 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 16:04:23.902500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1477/2913] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 16:04:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 16:04:33.900397 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=56 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 16:08:52.214385 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de6146-0e3c-4470-a7fb-2422d4ac650d service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 16:08:52.215780 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a1de6146-0e3c-4470-a7fb-2422d4ac650d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:08:52.301082 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de6146-0e3c-4470-a7fb-2422d4ac650d service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 16:08:52.306609 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588fd5c-af11-4396-9187-91a666862aa7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 16:08:52.307257 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588fd5c-af11-4396-9187-91a666862aa7 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 16:08:52.307546 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588fd5c-af11-4396-9187-91a666862aa7 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 16:08:52.307827 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588fd5c-af11-4396-9187-91a666862aa7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 16:08:52.317530 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588fd5c-af11-4396-9187-91a666862aa7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 16:08:52.319117 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588fd5c-af11-4396-9187-91a666862aa7 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 16:08:52.329143 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1437/2914] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 16:08:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 16:09:02.324794 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=37 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 16:09:22.745019 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceef563-9f4d-4e5b-91ef-43cb72f950c9 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 16:09:22.746566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-5ceef563-9f4d-4e5b-91ef-43cb72f950c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:09:22.777040 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceef563-9f4d-4e5b-91ef-43cb72f950c9 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 16:09:22.780643 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd118edc-0b07-42da-a9ae-88c4cc6bdb17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7245ef27d2454927a01a7cd56a4c27aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 16:09:22.781325 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd118edc-0b07-42da-a9ae-88c4cc6bdb17 service cinder] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 16:09:22.781614 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd118edc-0b07-42da-a9ae-88c4cc6bdb17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 16:09:22.781895 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd118edc-0b07-42da-a9ae-88c4cc6bdb17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 16:09:22.790752 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd118edc-0b07-42da-a9ae-88c4cc6bdb17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 16:09:22.792200 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd118edc-0b07-42da-a9ae-88c4cc6bdb17 service cinder] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 16:09:22.801288 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1478/2915] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 16:09:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 16:09:32.797043 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=15 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 16:10:20.525779 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588fd5c-af11-4396-9187-91a666862aa7 service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 16:10:20.527547 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4588fd5c-af11-4396-9187-91a666862aa7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:10:20.619826 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588fd5c-af11-4396-9187-91a666862aa7 service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 16:10:20.623520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5c0ac-e713-4fe9-81fa-618994df8f1a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '215c6f6f1aad486f85ce20590d889cbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 16:10:20.624430 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5c0ac-e713-4fe9-81fa-618994df8f1a service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 16:10:20.625527 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5c0ac-e713-4fe9-81fa-618994df8f1a service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 16:10:20.625844 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5c0ac-e713-4fe9-81fa-618994df8f1a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 16:10:20.627668 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9fa5c0ac-e713-4fe9-81fa-618994df8f1a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:10:20.661689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9fa5c0ac-e713-4fe9-81fa-618994df8f1a service glance] Could not find project: ae073bb1e3214abaabdf711901bfa192.: keystone.exception.ProjectNotFound: Could not find project: ae073bb1e3214abaabdf711901bfa192. Jul 19 16:10:20.661689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 16:10:20.661689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 16:10:20.661689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 16:10:20.661689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 16:10:20.661689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 16:10:20.661689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 16:10:20.661689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 16:10:20.661689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 16:10:20.661689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 16:10:20.661689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 16:10:20.661689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 16:10:20.661689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 19 16:10:20.661689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 19 16:10:20.661689 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 16:10:20.663158 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 16:10:20.664513 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 16:10:20.665817 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 16:10:20.667126 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: ae073bb1e3214abaabdf711901bfa192. Jul 19 16:10:20.667126 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 19 16:10:20.667126 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1438/2916] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 16:10:20 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 138 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Jul 19 16:10:30.631432 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=33 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 16:10:46.576775 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd118edc-0b07-42da-a9ae-88c4cc6bdb17 service cinder] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 16:10:46.578218 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-cd118edc-0b07-42da-a9ae-88c4cc6bdb17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:10:46.655652 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd118edc-0b07-42da-a9ae-88c4cc6bdb17 service cinder] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 16:10:46.659277 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7d35e-8985-499d-9f2c-e44b7ebbf1a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '369bdaded0534142b03ad3cf43a70881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 16:10:46.659923 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7d35e-8985-499d-9f2c-e44b7ebbf1a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 16:10:46.660204 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7d35e-8985-499d-9f2c-e44b7ebbf1a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 16:10:46.660480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7d35e-8985-499d-9f2c-e44b7ebbf1a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 16:10:46.662235 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-aba7d35e-8985-499d-9f2c-e44b7ebbf1a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:10:46.745973 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7d35e-8985-499d-9f2c-e44b7ebbf1a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 16:10:46.747396 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7d35e-8985-499d-9f2c-e44b7ebbf1a7 service neutron] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 16:10:46.766168 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1479/2917] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 16:10:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 16:10:47.198289 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5c0ac-e713-4fe9-81fa-618994df8f1a service glance] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 16:10:47.210635 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5c0ac-e713-4fe9-81fa-618994df8f1a service glance] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 16:10:47.215649 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23025fe3-2c6e-4399-b7d2-1c4af2b922f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 16:10:47.216290 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23025fe3-2c6e-4399-b7d2-1c4af2b922f2 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 16:10:47.216580 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23025fe3-2c6e-4399-b7d2-1c4af2b922f2 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 16:10:47.216835 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23025fe3-2c6e-4399-b7d2-1c4af2b922f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 16:10:47.218598 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-23025fe3-2c6e-4399-b7d2-1c4af2b922f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:10:47.232053 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23025fe3-2c6e-4399-b7d2-1c4af2b922f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 16:10:47.233894 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23025fe3-2c6e-4399-b7d2-1c4af2b922f2 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 16:10:47.249227 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1439/2918] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 16:10:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 16:10:47.265733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7d35e-8985-499d-9f2c-e44b7ebbf1a7 service neutron] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 16:10:47.267138 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-aba7d35e-8985-499d-9f2c-e44b7ebbf1a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:10:47.281493 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7d35e-8985-499d-9f2c-e44b7ebbf1a7 service neutron] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 16:10:47.286221 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0b9f9-3c55-4a97-9e81-cc179c1bb81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 16:10:47.287091 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0b9f9-3c55-4a97-9e81-cc179c1bb81a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 16:10:47.287373 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0b9f9-3c55-4a97-9e81-cc179c1bb81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 16:10:47.287645 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0b9f9-3c55-4a97-9e81-cc179c1bb81a admin admin] PATH_INFO: `/v3/users/746f7b2b60694c8a87a80f8801204345` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 16:10:47.303358 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0b9f9-3c55-4a97-9e81-cc179c1bb81a admin admin] RBAC: Authorizing `identity:delete_user(user_id=746f7b2b60694c8a87a80f8801204345)` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 16:10:47.305315 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0b9f9-3c55-4a97-9e81-cc179c1bb81a admin admin] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 16:10:47.348712 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 16:10:47.350480 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-37d0b9f9-3c55-4a97-9e81-cc179c1bb81a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '746f7b2b60694c8a87a80f8801204345'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:10:47.363837 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-37d0b9f9-3c55-4a97-9e81-cc179c1bb81a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '746f7b2b60694c8a87a80f8801204345'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:10:47.364500 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-37d0b9f9-3c55-4a97-9e81-cc179c1bb81a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '746f7b2b60694c8a87a80f8801204345'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:10:47.372170 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-37d0b9f9-3c55-4a97-9e81-cc179c1bb81a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '746f7b2b60694c8a87a80f8801204345'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:10:47.372781 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.notifications [None req-37d0b9f9-3c55-4a97-9e81-cc179c1bb81a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '746f7b2b60694c8a87a80f8801204345'} {{(pid=80271) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:10:47.385733 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1480/2919] 10.222.0.42 () {58 vars in 1223 bytes} [Tue Jul 19 16:10:47 2022] DELETE /identity/v3/users/746f7b2b60694c8a87a80f8801204345 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 16:10:47.558013 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23025fe3-2c6e-4399-b7d2-1c4af2b922f2 service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 16:10:47.568845 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23025fe3-2c6e-4399-b7d2-1c4af2b922f2 service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 16:10:47.628266 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d8740-0c8a-4ec2-a980-5b38d42140bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '669959f6d82b46b1b551b6c19bf21c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bb6648f966d4104a0136e3bb203cb76', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 16:10:47.629121 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d8740-0c8a-4ec2-a980-5b38d42140bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 16:10:47.629430 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d8740-0c8a-4ec2-a980-5b38d42140bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 16:10:47.629713 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d8740-0c8a-4ec2-a980-5b38d42140bc admin admin] PATH_INFO: `/v3/projects/a1e0f23ca7114ecea8330a41af830c22` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 16:10:47.630891 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d8740-0c8a-4ec2-a980-5b38d42140bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1e0f23ca7114ecea8330a41af830c22)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 16:10:47.632793 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d8740-0c8a-4ec2-a980-5b38d42140bc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 16:10:47.680019 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b5d8740-0c8a-4ec2-a980-5b38d42140bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1e0f23ca7114ecea8330a41af830c22', 'request_id': 'req-4b5d8740-0c8a-4ec2-a980-5b38d42140bc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:10:47.687190 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b5d8740-0c8a-4ec2-a980-5b38d42140bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1e0f23ca7114ecea8330a41af830c22', 'request_id': 'req-4b5d8740-0c8a-4ec2-a980-5b38d42140bc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:10:47.692688 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b5d8740-0c8a-4ec2-a980-5b38d42140bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1e0f23ca7114ecea8330a41af830c22', 'request_id': 'req-4b5d8740-0c8a-4ec2-a980-5b38d42140bc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:10:47.703520 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b5d8740-0c8a-4ec2-a980-5b38d42140bc admin admin] The token cache is being invalidate because project a1e0f23ca7114ecea8330a41af830c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 16:10:47.703806 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b5d8740-0c8a-4ec2-a980-5b38d42140bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:10:47.704419 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4b5d8740-0c8a-4ec2-a980-5b38d42140bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 16:10:47.705957 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1440/2920] 10.222.0.42 () {58 vars in 1232 bytes} [Tue Jul 19 16:10:47 2022] DELETE /identity/v3/projects/a1e0f23ca7114ecea8330a41af830c22 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 16:10:57.381724 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=13,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 16:10:57.700620 dv-cinder-849846-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 16:13:57.138702 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0b9f9-3c55-4a97-9e81-cc179c1bb81a admin admin] Authenticating user token {{(pid=80271) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 16:13:57.140159 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-37d0b9f9-3c55-4a97-9e81-cc179c1bb81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:13:57.208517 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0b9f9-3c55-4a97-9e81-cc179c1bb81a admin admin] Validating token access rules against request {{(pid=80271) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 16:13:57.272777 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb0474-8fcf-4068-98a2-41ec6c27c277 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6002f32936944b0db6fd04f3851836db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f51e47b8e34bf5ac822d8c93853e30', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80271) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 16:13:57.273566 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb0474-8fcf-4068-98a2-41ec6c27c277 service placement] REQUEST_METHOD: `GET` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 16:13:57.273856 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb0474-8fcf-4068-98a2-41ec6c27c277 service placement] SCRIPT_NAME: `/identity` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 16:13:57.274137 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb0474-8fcf-4068-98a2-41ec6c27c277 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80271) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 16:13:57.275920 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.fernet_utils [None req-41eb0474-8fcf-4068-98a2-41ec6c27c277 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80271) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 16:13:57.287260 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb0474-8fcf-4068-98a2-41ec6c27c277 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 16:13:57.288998 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb0474-8fcf-4068-98a2-41ec6c27c277 service placement] RBAC: Authorization granted {{(pid=80271) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 16:13:57.298913 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: [pid: 80271|app: 0|req: 1481/2921] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Jul 19 16:13:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 16:14:07.294569 dv-cinder-849846-2-iscsi devstack@keystone.service[80271]: DEBUG dbcounter [-] [80271] Writing DB stats keystone:SELECT=58 {{(pid=80271) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}